{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retriever Customization - Synthetic Data Generation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Authors - Aditya Malte, Vinay Raman"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "Text Retriever/Embedding models serve the purpose of allowing the retrieval of a relevant document given a query. However, they might not perform as well, if the target documents/queries are out-of-domain for the retriever. An example includes retrieving medical documents using a retriever trained only on generic text."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In these cases, it is often the case that a domain-specific training data must be created to 'adapt' these retrievers. This is often a time-consuming and costly process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook is a sample demonstration on how Large Language Models (LLMs) could be used to synthetically generate training data, which can then be used to adapt retriever models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the first notebook as part of a two-notebook tutorial. The end-goal of this tutorial is a demo on:\n",
    "1. `synthetic_data_generation_nemo.ipynb`(this notebook) Generating Synthetic Training Data - training examples containing generated queries and positive chunks. The results of which are saved to a `.csv` file, the path of which is printed at the end\n",
    "2. `retriever_customization.ipynb` - Using the generated training data in the `.csv` file to fine-tune a retriever model\n",
    "\n",
    "NOTE: This tutorial is only meant as a demo, and hence only a small subset of the corpus is used for training data generation - in order for the notebook run to complete in a reasonable time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Instructions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. This notebook runs in a Docker environment built from the NeMo FW repo. Refer https://github.com/NVIDIA/NeMo/tree/main for instructions on how to build and run the docker containers. Ensure that the docker container you run this notebook in is built from the `main` branch of the `NeMo` repository.\n",
    "2. Update you HuggingFace token to download the Llama model using `export HUGGINGFACE_TOKEN=<yourtoken>`\n",
    "3. Before running this notebook, please run setup.sh to perform the following functions:\n",
    "    1. Download the Llama-2-13B Chat model from HuggingFace\n",
    "    2. Convert it into a .nemo checkpoint\n",
    "    3. Serve the converted model in a Megatron Server\n",
    "    \n",
    "This notebook was tested on a setup comprising 2xA6000 GPUs with CUDA setup. Please ensure you have adequate GPU memory to avoid CUDA out-of-memory issues."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import numpy as np\n",
    "import os\n",
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from tqdm import tqdm\n",
    "import re\n",
    "import datetime\n",
    "import re\n",
    "from nltk.tokenize import sent_tokenize\n",
    "import nltk\n",
    "nltk.download('punkt')\n",
    "from sklearn.model_selection import train_test_split\n",
    "import requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from datasets import load_dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an example, we have chosen the `nfcorpus` to generate training data. You may choose a corpus of your domain, or even feed your own domain data for this process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "DOMAIN = \"BeIR/nfcorpus\"\n",
    "corpus = load_dataset(DOMAIN, \"corpus\")[\"corpus\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "GENERATIONS_SAVE_DIR = \"/workspace/files/data\"\n",
    "GENERATION_MODEL_NAME_OR_PATH = \"meta-llama/Llama-2-13b-chat-hf\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "DO_SAMPLE=True\n",
    "TOKENS_TO_GENERATE = 384\n",
    "TEMPERATURE = 0.2\n",
    "TOP_P= 0.95\n",
    "TOP_K= 50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Synthetic Data Generation from Knowledge Base"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chunk Knowledge Base"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Chunking is required to break large documents into smaller chunks that an LLM can take as input. I this case we chunk the texts into samples of around word count 300, ensuring that sentences are not broken. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chunk_text(samples, chunk_size=300):\n",
    "    \n",
    "    final_chunks = []\n",
    "    for idx, (text, title, paragraph_id) in enumerate(samples):\n",
    "        sentence_list = sent_tokenize(text)\n",
    "        chunk = []\n",
    "        chunk_id = 0\n",
    "        word_count = 0\n",
    "        for sentence in sentence_list:\n",
    "            word_tokens = sentence.split()\n",
    "            word_count += len(word_tokens)\n",
    "            chunk.append(sentence)\n",
    "            \n",
    "            if(word_count >= chunk_size):\n",
    "                chunk_text = \" \".join(chunk)\n",
    "                final_chunks.append((chunk_text, title, paragraph_id, chunk_id))\n",
    "                chunk_id += 1\n",
    "                \n",
    "                chunk = []\n",
    "                word_count = 0\n",
    "                \n",
    "        if len(chunk) > 0:\n",
    "            chunk_text = \" \".join(chunk)\n",
    "            if(chunk_id==0 or len(chunk_text.split())>int((0.4*chunk_size))): # Only include the last chunk if it has significant number of words\n",
    "                final_chunks.append((chunk_text, title, paragraph_id, chunk_id)) # , or if the sample itself was a single chunk (chunk_id=0)\n",
    "\n",
    "    return final_chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>_id</th>\n",
       "      <th>title</th>\n",
       "      <th>text</th>\n",
       "      <th>paragraph_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MED-10</td>\n",
       "      <td>Statin Use and Breast Cancer Survival: A Natio...</td>\n",
       "      <td>Recent studies have suggested that statins, an...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MED-14</td>\n",
       "      <td>Statin use after diagnosis of breast cancer an...</td>\n",
       "      <td>BACKGROUND: Preclinical studies have shown tha...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MED-118</td>\n",
       "      <td>Alkylphenols in human milk and their relations...</td>\n",
       "      <td>The aims of this study were to determine the c...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MED-301</td>\n",
       "      <td>Methylmercury: A Potential Environmental Risk ...</td>\n",
       "      <td>Epilepsy or seizure disorder is one of the mos...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MED-306</td>\n",
       "      <td>Sensitivity of Continuous Performance Test (CP...</td>\n",
       "      <td>Hit Reaction Time latencies (HRT) in the Conti...</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3628</th>\n",
       "      <td>MED-917</td>\n",
       "      <td>Effect of freezing and storage on the phenolic...</td>\n",
       "      <td>Scottish-grown red raspberries are a rich sour...</td>\n",
       "      <td>3628</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3629</th>\n",
       "      <td>MED-941</td>\n",
       "      <td>Topical vitamin A treatment of recalcitrant co...</td>\n",
       "      <td>BACKGROUND: Common warts (verruca vulgaris) ar...</td>\n",
       "      <td>3629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3630</th>\n",
       "      <td>MED-942</td>\n",
       "      <td>Esophageal injury by apple cider vinegar table...</td>\n",
       "      <td>Apple cider vinegar products are advertised in...</td>\n",
       "      <td>3630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3631</th>\n",
       "      <td>MED-952</td>\n",
       "      <td>Cannabis and the lung.</td>\n",
       "      <td>The use of cannabis is embedded within many so...</td>\n",
       "      <td>3631</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3632</th>\n",
       "      <td>MED-961</td>\n",
       "      <td>Vitamin D(3) is more potent than vitamin D(2) ...</td>\n",
       "      <td>BACKGROUND: Current unitage for the calciferol...</td>\n",
       "      <td>3632</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3633 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          _id                                              title  \\\n",
       "0      MED-10  Statin Use and Breast Cancer Survival: A Natio...   \n",
       "1      MED-14  Statin use after diagnosis of breast cancer an...   \n",
       "2     MED-118  Alkylphenols in human milk and their relations...   \n",
       "3     MED-301  Methylmercury: A Potential Environmental Risk ...   \n",
       "4     MED-306  Sensitivity of Continuous Performance Test (CP...   \n",
       "...       ...                                                ...   \n",
       "3628  MED-917  Effect of freezing and storage on the phenolic...   \n",
       "3629  MED-941  Topical vitamin A treatment of recalcitrant co...   \n",
       "3630  MED-942  Esophageal injury by apple cider vinegar table...   \n",
       "3631  MED-952                             Cannabis and the lung.   \n",
       "3632  MED-961  Vitamin D(3) is more potent than vitamin D(2) ...   \n",
       "\n",
       "                                                   text  paragraph_id  \n",
       "0     Recent studies have suggested that statins, an...             0  \n",
       "1     BACKGROUND: Preclinical studies have shown tha...             1  \n",
       "2     The aims of this study were to determine the c...             2  \n",
       "3     Epilepsy or seizure disorder is one of the mos...             3  \n",
       "4     Hit Reaction Time latencies (HRT) in the Conti...             4  \n",
       "...                                                 ...           ...  \n",
       "3628  Scottish-grown red raspberries are a rich sour...          3628  \n",
       "3629  BACKGROUND: Common warts (verruca vulgaris) ar...          3629  \n",
       "3630  Apple cider vinegar products are advertised in...          3630  \n",
       "3631  The use of cannabis is embedded within many so...          3631  \n",
       "3632  BACKGROUND: Current unitage for the calciferol...          3632  \n",
       "\n",
       "[3633 rows x 4 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kb = pd.DataFrame(corpus)\n",
    "kb[\"paragraph_id\"] = range(len(kb))\n",
    "kb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As this is just an example notebook, we are only sampling 100 out of around 5000 documents in the corpus, in order to allow the notebook to complete in a reasonable time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>_id</th>\n",
       "      <th>title</th>\n",
       "      <th>text</th>\n",
       "      <th>paragraph_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2386</th>\n",
       "      <td>MED-3931</td>\n",
       "      <td>Pilot dietary study with normoproteic protein-...</td>\n",
       "      <td>Although a plant-based diet can provide some b...</td>\n",
       "      <td>2386</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2546</th>\n",
       "      <td>MED-4147</td>\n",
       "      <td>Occurrence of steroid hormones and antibiotics...</td>\n",
       "      <td>Wastewater impoundments at concentrated animal...</td>\n",
       "      <td>2546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1296</th>\n",
       "      <td>MED-2335</td>\n",
       "      <td>Xenohormesis: health benefits from an eon of p...</td>\n",
       "      <td>Xenohormesis is a biological principle that ex...</td>\n",
       "      <td>1296</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3369</th>\n",
       "      <td>MED-5120</td>\n",
       "      <td>L-theanine intervention enhances human gammade...</td>\n",
       "      <td>Human gammadeltaT lymphocytes are a subset of ...</td>\n",
       "      <td>3369</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>202</th>\n",
       "      <td>MED-1002</td>\n",
       "      <td>Polybrominated diphenyl ethers (PBDEs), hydrox...</td>\n",
       "      <td>Prenatal exposure to polybrominated diphenyl e...</td>\n",
       "      <td>202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1829</th>\n",
       "      <td>MED-3179</td>\n",
       "      <td>Cognitive impairment and dementia in neurocyst...</td>\n",
       "      <td>OBJECTIVES: Neurocysticercosis (NCYST) is the ...</td>\n",
       "      <td>1829</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2787</th>\n",
       "      <td>MED-4462</td>\n",
       "      <td>Sulforaphane protects human chondrocytes again...</td>\n",
       "      <td>Chondrocyte cell death can contribute to carti...</td>\n",
       "      <td>2787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2450</th>\n",
       "      <td>MED-4007</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>2450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2862</th>\n",
       "      <td>MED-4550</td>\n",
       "      <td>An impact of the diet on serum fatty acid and ...</td>\n",
       "      <td>BACKGROUND/OBJECTIVES: Vegetarian diet has bec...</td>\n",
       "      <td>2862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2703</th>\n",
       "      <td>MED-4361</td>\n",
       "      <td>Dirty money: an investigation into the hygiene...</td>\n",
       "      <td>A total of 1280 banknotes were obtained from f...</td>\n",
       "      <td>2703</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           _id                                              title  \\\n",
       "2386  MED-3931  Pilot dietary study with normoproteic protein-...   \n",
       "2546  MED-4147  Occurrence of steroid hormones and antibiotics...   \n",
       "1296  MED-2335  Xenohormesis: health benefits from an eon of p...   \n",
       "3369  MED-5120  L-theanine intervention enhances human gammade...   \n",
       "202   MED-1002  Polybrominated diphenyl ethers (PBDEs), hydrox...   \n",
       "...        ...                                                ...   \n",
       "1829  MED-3179  Cognitive impairment and dementia in neurocyst...   \n",
       "2787  MED-4462  Sulforaphane protects human chondrocytes again...   \n",
       "2450  MED-4007  Study of the ketogenic agent AC-1202 in mild t...   \n",
       "2862  MED-4550  An impact of the diet on serum fatty acid and ...   \n",
       "2703  MED-4361  Dirty money: an investigation into the hygiene...   \n",
       "\n",
       "                                                   text  paragraph_id  \n",
       "2386  Although a plant-based diet can provide some b...          2386  \n",
       "2546  Wastewater impoundments at concentrated animal...          2546  \n",
       "1296  Xenohormesis is a biological principle that ex...          1296  \n",
       "3369  Human gammadeltaT lymphocytes are a subset of ...          3369  \n",
       "202   Prenatal exposure to polybrominated diphenyl e...           202  \n",
       "...                                                 ...           ...  \n",
       "1829  OBJECTIVES: Neurocysticercosis (NCYST) is the ...          1829  \n",
       "2787  Chondrocyte cell death can contribute to carti...          2787  \n",
       "2450  Background Alzheimer's disease (AD) is charact...          2450  \n",
       "2862  BACKGROUND/OBJECTIVES: Vegetarian diet has bec...          2862  \n",
       "2703  A total of 1280 banknotes were obtained from f...          2703  \n",
       "\n",
       "[100 rows x 4 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kb = kb.sample(n=100)\n",
    "kb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>chunk_text</th>\n",
       "      <th>paragraph_id</th>\n",
       "      <th>chunk_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Although a plant-based diet can provide some b...</td>\n",
       "      <td>2386</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Wastewater impoundments at concentrated animal...</td>\n",
       "      <td>2546</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Xenohormesis is a biological principle that ex...</td>\n",
       "      <td>1296</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Human gammadeltaT lymphocytes are a subset of ...</td>\n",
       "      <td>3369</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Prenatal exposure to polybrominated diphenyl e...</td>\n",
       "      <td>202</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>Chondrocyte cell death can contribute to carti...</td>\n",
       "      <td>2787</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>2450</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>In the per protocol population, E4(-) particip...</td>\n",
       "      <td>2450</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>BACKGROUND/OBJECTIVES: Vegetarian diet has bec...</td>\n",
       "      <td>2862</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>A total of 1280 banknotes were obtained from f...</td>\n",
       "      <td>2703</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>101 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            chunk_text  paragraph_id  chunk_id\n",
       "0    Although a plant-based diet can provide some b...          2386         0\n",
       "1    Wastewater impoundments at concentrated animal...          2546         0\n",
       "2    Xenohormesis is a biological principle that ex...          1296         0\n",
       "3    Human gammadeltaT lymphocytes are a subset of ...          3369         0\n",
       "4    Prenatal exposure to polybrominated diphenyl e...           202         0\n",
       "..                                                 ...           ...       ...\n",
       "96   Chondrocyte cell death can contribute to carti...          2787         0\n",
       "97   Background Alzheimer's disease (AD) is charact...          2450         0\n",
       "98   In the per protocol population, E4(-) particip...          2450         1\n",
       "99   BACKGROUND/OBJECTIVES: Vegetarian diet has bec...          2862         0\n",
       "100  A total of 1280 banknotes were obtained from f...          2703         0\n",
       "\n",
       "[101 rows x 3 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kb_chunked = pd.DataFrame(chunk_text(kb[[\"text\", \"title\", \"paragraph_id\"]].itertuples(index=False)), columns=[\"text\", \"title\", \"paragraph_id\", \"chunk_id\"])\n",
    "kb_chunked.columns = ['chunk_text', 'title', 'paragraph_id', 'chunk_id']\n",
    "kb_chunked.drop([\"title\"], axis=1, inplace=True)\n",
    "kb_chunked"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>_id</th>\n",
       "      <th>title</th>\n",
       "      <th>text</th>\n",
       "      <th>paragraph_id</th>\n",
       "      <th>chunk_text</th>\n",
       "      <th>chunk_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MED-3931</td>\n",
       "      <td>Pilot dietary study with normoproteic protein-...</td>\n",
       "      <td>Although a plant-based diet can provide some b...</td>\n",
       "      <td>2386</td>\n",
       "      <td>Although a plant-based diet can provide some b...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MED-4147</td>\n",
       "      <td>Occurrence of steroid hormones and antibiotics...</td>\n",
       "      <td>Wastewater impoundments at concentrated animal...</td>\n",
       "      <td>2546</td>\n",
       "      <td>Wastewater impoundments at concentrated animal...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MED-2335</td>\n",
       "      <td>Xenohormesis: health benefits from an eon of p...</td>\n",
       "      <td>Xenohormesis is a biological principle that ex...</td>\n",
       "      <td>1296</td>\n",
       "      <td>Xenohormesis is a biological principle that ex...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MED-5120</td>\n",
       "      <td>L-theanine intervention enhances human gammade...</td>\n",
       "      <td>Human gammadeltaT lymphocytes are a subset of ...</td>\n",
       "      <td>3369</td>\n",
       "      <td>Human gammadeltaT lymphocytes are a subset of ...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MED-1002</td>\n",
       "      <td>Polybrominated diphenyl ethers (PBDEs), hydrox...</td>\n",
       "      <td>Prenatal exposure to polybrominated diphenyl e...</td>\n",
       "      <td>202</td>\n",
       "      <td>Prenatal exposure to polybrominated diphenyl e...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>MED-4462</td>\n",
       "      <td>Sulforaphane protects human chondrocytes again...</td>\n",
       "      <td>Chondrocyte cell death can contribute to carti...</td>\n",
       "      <td>2787</td>\n",
       "      <td>Chondrocyte cell death can contribute to carti...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>MED-4007</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>2450</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>MED-4007</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>2450</td>\n",
       "      <td>In the per protocol population, E4(-) particip...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>MED-4550</td>\n",
       "      <td>An impact of the diet on serum fatty acid and ...</td>\n",
       "      <td>BACKGROUND/OBJECTIVES: Vegetarian diet has bec...</td>\n",
       "      <td>2862</td>\n",
       "      <td>BACKGROUND/OBJECTIVES: Vegetarian diet has bec...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>MED-4361</td>\n",
       "      <td>Dirty money: an investigation into the hygiene...</td>\n",
       "      <td>A total of 1280 banknotes were obtained from f...</td>\n",
       "      <td>2703</td>\n",
       "      <td>A total of 1280 banknotes were obtained from f...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>101 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          _id                                              title  \\\n",
       "0    MED-3931  Pilot dietary study with normoproteic protein-...   \n",
       "1    MED-4147  Occurrence of steroid hormones and antibiotics...   \n",
       "2    MED-2335  Xenohormesis: health benefits from an eon of p...   \n",
       "3    MED-5120  L-theanine intervention enhances human gammade...   \n",
       "4    MED-1002  Polybrominated diphenyl ethers (PBDEs), hydrox...   \n",
       "..        ...                                                ...   \n",
       "96   MED-4462  Sulforaphane protects human chondrocytes again...   \n",
       "97   MED-4007  Study of the ketogenic agent AC-1202 in mild t...   \n",
       "98   MED-4007  Study of the ketogenic agent AC-1202 in mild t...   \n",
       "99   MED-4550  An impact of the diet on serum fatty acid and ...   \n",
       "100  MED-4361  Dirty money: an investigation into the hygiene...   \n",
       "\n",
       "                                                  text  paragraph_id  \\\n",
       "0    Although a plant-based diet can provide some b...          2386   \n",
       "1    Wastewater impoundments at concentrated animal...          2546   \n",
       "2    Xenohormesis is a biological principle that ex...          1296   \n",
       "3    Human gammadeltaT lymphocytes are a subset of ...          3369   \n",
       "4    Prenatal exposure to polybrominated diphenyl e...           202   \n",
       "..                                                 ...           ...   \n",
       "96   Chondrocyte cell death can contribute to carti...          2787   \n",
       "97   Background Alzheimer's disease (AD) is charact...          2450   \n",
       "98   Background Alzheimer's disease (AD) is charact...          2450   \n",
       "99   BACKGROUND/OBJECTIVES: Vegetarian diet has bec...          2862   \n",
       "100  A total of 1280 banknotes were obtained from f...          2703   \n",
       "\n",
       "                                            chunk_text  chunk_id  \n",
       "0    Although a plant-based diet can provide some b...         0  \n",
       "1    Wastewater impoundments at concentrated animal...         0  \n",
       "2    Xenohormesis is a biological principle that ex...         0  \n",
       "3    Human gammadeltaT lymphocytes are a subset of ...         0  \n",
       "4    Prenatal exposure to polybrominated diphenyl e...         0  \n",
       "..                                                 ...       ...  \n",
       "96   Chondrocyte cell death can contribute to carti...         0  \n",
       "97   Background Alzheimer's disease (AD) is charact...         0  \n",
       "98   In the per protocol population, E4(-) particip...         1  \n",
       "99   BACKGROUND/OBJECTIVES: Vegetarian diet has bec...         0  \n",
       "100  A total of 1280 banknotes were obtained from f...         0  \n",
       "\n",
       "[101 rows x 6 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kb = kb.merge(kb_chunked, how=\"left\", on=\"paragraph_id\")\n",
    "kb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>_id</th>\n",
       "      <th>title</th>\n",
       "      <th>text</th>\n",
       "      <th>paragraph_id</th>\n",
       "      <th>chunk_text</th>\n",
       "      <th>chunk_id</th>\n",
       "      <th>title_text_concat</th>\n",
       "      <th>chunk_title_text_concat</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MED-3931</td>\n",
       "      <td>Pilot dietary study with normoproteic protein-...</td>\n",
       "      <td>Although a plant-based diet can provide some b...</td>\n",
       "      <td>2386</td>\n",
       "      <td>Although a plant-based diet can provide some b...</td>\n",
       "      <td>0</td>\n",
       "      <td>Pilot dietary study with normoproteic protein-...</td>\n",
       "      <td>Pilot dietary study with normoproteic protein-...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MED-4147</td>\n",
       "      <td>Occurrence of steroid hormones and antibiotics...</td>\n",
       "      <td>Wastewater impoundments at concentrated animal...</td>\n",
       "      <td>2546</td>\n",
       "      <td>Wastewater impoundments at concentrated animal...</td>\n",
       "      <td>0</td>\n",
       "      <td>Occurrence of steroid hormones and antibiotics...</td>\n",
       "      <td>Occurrence of steroid hormones and antibiotics...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MED-2335</td>\n",
       "      <td>Xenohormesis: health benefits from an eon of p...</td>\n",
       "      <td>Xenohormesis is a biological principle that ex...</td>\n",
       "      <td>1296</td>\n",
       "      <td>Xenohormesis is a biological principle that ex...</td>\n",
       "      <td>0</td>\n",
       "      <td>Xenohormesis: health benefits from an eon of p...</td>\n",
       "      <td>Xenohormesis: health benefits from an eon of p...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MED-5120</td>\n",
       "      <td>L-theanine intervention enhances human gammade...</td>\n",
       "      <td>Human gammadeltaT lymphocytes are a subset of ...</td>\n",
       "      <td>3369</td>\n",
       "      <td>Human gammadeltaT lymphocytes are a subset of ...</td>\n",
       "      <td>0</td>\n",
       "      <td>L-theanine intervention enhances human gammade...</td>\n",
       "      <td>L-theanine intervention enhances human gammade...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MED-1002</td>\n",
       "      <td>Polybrominated diphenyl ethers (PBDEs), hydrox...</td>\n",
       "      <td>Prenatal exposure to polybrominated diphenyl e...</td>\n",
       "      <td>202</td>\n",
       "      <td>Prenatal exposure to polybrominated diphenyl e...</td>\n",
       "      <td>0</td>\n",
       "      <td>Polybrominated diphenyl ethers (PBDEs), hydrox...</td>\n",
       "      <td>Polybrominated diphenyl ethers (PBDEs), hydrox...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>MED-4462</td>\n",
       "      <td>Sulforaphane protects human chondrocytes again...</td>\n",
       "      <td>Chondrocyte cell death can contribute to carti...</td>\n",
       "      <td>2787</td>\n",
       "      <td>Chondrocyte cell death can contribute to carti...</td>\n",
       "      <td>0</td>\n",
       "      <td>Sulforaphane protects human chondrocytes again...</td>\n",
       "      <td>Sulforaphane protects human chondrocytes again...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>MED-4007</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>2450</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>0</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>MED-4007</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>2450</td>\n",
       "      <td>In the per protocol population, E4(-) particip...</td>\n",
       "      <td>1</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>MED-4550</td>\n",
       "      <td>An impact of the diet on serum fatty acid and ...</td>\n",
       "      <td>BACKGROUND/OBJECTIVES: Vegetarian diet has bec...</td>\n",
       "      <td>2862</td>\n",
       "      <td>BACKGROUND/OBJECTIVES: Vegetarian diet has bec...</td>\n",
       "      <td>0</td>\n",
       "      <td>An impact of the diet on serum fatty acid and ...</td>\n",
       "      <td>An impact of the diet on serum fatty acid and ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>MED-4361</td>\n",
       "      <td>Dirty money: an investigation into the hygiene...</td>\n",
       "      <td>A total of 1280 banknotes were obtained from f...</td>\n",
       "      <td>2703</td>\n",
       "      <td>A total of 1280 banknotes were obtained from f...</td>\n",
       "      <td>0</td>\n",
       "      <td>Dirty money: an investigation into the hygiene...</td>\n",
       "      <td>Dirty money: an investigation into the hygiene...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>101 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          _id                                              title  \\\n",
       "0    MED-3931  Pilot dietary study with normoproteic protein-...   \n",
       "1    MED-4147  Occurrence of steroid hormones and antibiotics...   \n",
       "2    MED-2335  Xenohormesis: health benefits from an eon of p...   \n",
       "3    MED-5120  L-theanine intervention enhances human gammade...   \n",
       "4    MED-1002  Polybrominated diphenyl ethers (PBDEs), hydrox...   \n",
       "..        ...                                                ...   \n",
       "96   MED-4462  Sulforaphane protects human chondrocytes again...   \n",
       "97   MED-4007  Study of the ketogenic agent AC-1202 in mild t...   \n",
       "98   MED-4007  Study of the ketogenic agent AC-1202 in mild t...   \n",
       "99   MED-4550  An impact of the diet on serum fatty acid and ...   \n",
       "100  MED-4361  Dirty money: an investigation into the hygiene...   \n",
       "\n",
       "                                                  text  paragraph_id  \\\n",
       "0    Although a plant-based diet can provide some b...          2386   \n",
       "1    Wastewater impoundments at concentrated animal...          2546   \n",
       "2    Xenohormesis is a biological principle that ex...          1296   \n",
       "3    Human gammadeltaT lymphocytes are a subset of ...          3369   \n",
       "4    Prenatal exposure to polybrominated diphenyl e...           202   \n",
       "..                                                 ...           ...   \n",
       "96   Chondrocyte cell death can contribute to carti...          2787   \n",
       "97   Background Alzheimer's disease (AD) is charact...          2450   \n",
       "98   Background Alzheimer's disease (AD) is charact...          2450   \n",
       "99   BACKGROUND/OBJECTIVES: Vegetarian diet has bec...          2862   \n",
       "100  A total of 1280 banknotes were obtained from f...          2703   \n",
       "\n",
       "                                            chunk_text  chunk_id  \\\n",
       "0    Although a plant-based diet can provide some b...         0   \n",
       "1    Wastewater impoundments at concentrated animal...         0   \n",
       "2    Xenohormesis is a biological principle that ex...         0   \n",
       "3    Human gammadeltaT lymphocytes are a subset of ...         0   \n",
       "4    Prenatal exposure to polybrominated diphenyl e...         0   \n",
       "..                                                 ...       ...   \n",
       "96   Chondrocyte cell death can contribute to carti...         0   \n",
       "97   Background Alzheimer's disease (AD) is charact...         0   \n",
       "98   In the per protocol population, E4(-) particip...         1   \n",
       "99   BACKGROUND/OBJECTIVES: Vegetarian diet has bec...         0   \n",
       "100  A total of 1280 banknotes were obtained from f...         0   \n",
       "\n",
       "                                     title_text_concat  \\\n",
       "0    Pilot dietary study with normoproteic protein-...   \n",
       "1    Occurrence of steroid hormones and antibiotics...   \n",
       "2    Xenohormesis: health benefits from an eon of p...   \n",
       "3    L-theanine intervention enhances human gammade...   \n",
       "4    Polybrominated diphenyl ethers (PBDEs), hydrox...   \n",
       "..                                                 ...   \n",
       "96   Sulforaphane protects human chondrocytes again...   \n",
       "97   Study of the ketogenic agent AC-1202 in mild t...   \n",
       "98   Study of the ketogenic agent AC-1202 in mild t...   \n",
       "99   An impact of the diet on serum fatty acid and ...   \n",
       "100  Dirty money: an investigation into the hygiene...   \n",
       "\n",
       "                               chunk_title_text_concat  \n",
       "0    Pilot dietary study with normoproteic protein-...  \n",
       "1    Occurrence of steroid hormones and antibiotics...  \n",
       "2    Xenohormesis: health benefits from an eon of p...  \n",
       "3    L-theanine intervention enhances human gammade...  \n",
       "4    Polybrominated diphenyl ethers (PBDEs), hydrox...  \n",
       "..                                                 ...  \n",
       "96   Sulforaphane protects human chondrocytes again...  \n",
       "97   Study of the ketogenic agent AC-1202 in mild t...  \n",
       "98   Study of the ketogenic agent AC-1202 in mild t...  \n",
       "99   An impact of the diet on serum fatty acid and ...  \n",
       "100  Dirty money: an investigation into the hygiene...  \n",
       "\n",
       "[101 rows x 8 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kb[\"title_text_concat\"] = kb[\"title\"] + \"\\n\" + kb[\"text\"]\n",
    "kb[\"chunk_title_text_concat\"] = kb[\"title\"] + \"\\n\" + kb[\"chunk_text\"]\n",
    "kb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prompt Generation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A prompt serves the purpose of providing context to the LLM for generation. You could modify this prompt as per your desired domain\n",
    "Some ideas including modifying the example to suit your domain\n",
    "The current prompt is used as an examplea and has been picked up from NVIDIA documentation/help page\n",
    "You could modify the prompt to represent your desired domain. The queries you give as an example  define the kind of queries the model will generate. In this prompt we ask the LLM to generate three unique questions.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You are a data annotator trying to generate three search queries for the Document 2. The generated queries must be answerable from Document 2. Each generated query must be enclosed within the <q> and </q> tags as shown in Example. Only generate the query, do not generate the answer. An example is:\n",
      "Example:AV Sync \n",
      "Use of an AV Receiver with HDMI for video may result in audio lagging behind video.  First try \n",
      "using the receiver AV sync settings to calibrate.  If this does not work, use the AV sync slider \n",
      "utility in Settings  > Display & sound > Advanced settings > Audio video sync to calibrate for \n",
      "any audio delay.  The AV sync slider allows you to advance audio by 1 second (in small \n",
      "increments of 10ms) to synchronize the audio and video. \n",
      "Note that this tool is effective only when SHIELD is connected to your AV Receiver over HDMI \n",
      "(i.e. audio/video over HDMI); it is not meant to be used when a headset is plugged into SHIELD \n",
      "Controller/SHIELD Remote or USB audio device or Bluetooth audio device. \n",
      "If video lags behind audio (i.e. audio is ahead of video) then use your AV receiver’s settings to \n",
      "delay audio.\n",
      "ADJUST FOR OVERSCAN\n",
      "\n",
      "For TVs that don't provide their own overscan settings, use this setting to adjust the picture size to fit the screen.\n",
      "\n",
      "Go to Settings > Device Preferences > Display & Sound > Advanced Settings > Adjust for overscan to resize the picture on your TV or display.  Use the UP and DOWN d-pad buttons on your remote to maximize the picture on your TV.  Make sure the green triangles are completely visible to avoid overscan.\n",
      "Generated Queries:\n",
      "1. <q>How do I adjust the display so that my picture does not go out of the screen?</q>\n",
      "2. <q>Why is AV Sync not working when I'm plugging my SHIELD into my bluetooth earphone?</q>\n",
      "3. <q>How many seconds can I delay audio by in AV Sync?</q>\n",
      "Do not use text from Example to generate queries for Document 2.\n"
     ]
    }
   ],
   "source": [
    "system_prompt = \"\"\"You are a data annotator trying to generate three search queries for the Document 2. The generated queries must be answerable from Document 2. Each generated query must be enclosed within the <q> and </q> tags as shown in Example. Only generate the query, do not generate the answer. An example is:\\n\"\"\"\n",
    "\n",
    "example = \"\"\"Example:AV Sync \n",
    "Use of an AV Receiver with HDMI for video may result in audio lagging behind video.  First try \n",
    "using the receiver AV sync settings to calibrate.  If this does not work, use the AV sync slider \n",
    "utility in Settings  > Display & sound > Advanced settings > Audio video sync to calibrate for \n",
    "any audio delay.  The AV sync slider allows you to advance audio by 1 second (in small \n",
    "increments of 10ms) to synchronize the audio and video. \n",
    "Note that this tool is effective only when SHIELD is connected to your AV Receiver over HDMI \n",
    "(i.e. audio/video over HDMI); it is not meant to be used when a headset is plugged into SHIELD \n",
    "Controller/SHIELD Remote or USB audio device or Bluetooth audio device. \n",
    "If video lags behind audio (i.e. audio is ahead of video) then use your AV receiver’s settings to \n",
    "delay audio.\n",
    "ADJUST FOR OVERSCAN\n",
    "\n",
    "For TVs that don't provide their own overscan settings, use this setting to adjust the picture size to fit the screen.\n",
    "\n",
    "Go to Settings > Device Preferences > Display & Sound > Advanced Settings > Adjust for overscan to resize the picture on your TV or display.  Use the UP and DOWN d-pad buttons on your remote to maximize the picture on your TV.  Make sure the green triangles are completely visible to avoid overscan.\n",
    "Generated Queries:\n",
    "1. <q>How do I adjust the display so that my picture does not go out of the screen?</q>\n",
    "2. <q>Why is AV Sync not working when I'm plugging my SHIELD into my bluetooth earphone?</q>\n",
    "3. <q>How many seconds can I delay audio by in AV Sync?</q>\n",
    "\"\"\"\n",
    "system_prompt += example\n",
    "system_prompt += \"Do not use text from Example to generate queries for Document 2.\"\n",
    "print(system_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def get_prompt(message: str, chat_history: list[tuple[str, str]],\n",
    "               system_prompt: str) -> str:\n",
    "    texts = [f'<s>[INST] <<SYS>>\\n{system_prompt}\\n<</SYS>>\\n\\n']\n",
    "    # The first user input is _not_ stripped\n",
    "    do_strip = False\n",
    "    for user_input, response in chat_history:\n",
    "        user_input = user_input.strip() if do_strip else user_input\n",
    "        do_strip = True\n",
    "        texts.append(f'{user_input} [/INST] {response.strip()} </s><s>[INST] ') \n",
    "    message = message.strip() if do_strip else message\n",
    "    message = \"\\nDocument 2:\" + message\n",
    "    texts.append(f'{message} [/INST]')\n",
    "    return ''.join(texts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>_id</th>\n",
       "      <th>title</th>\n",
       "      <th>text</th>\n",
       "      <th>paragraph_id</th>\n",
       "      <th>chunk_text</th>\n",
       "      <th>chunk_id</th>\n",
       "      <th>title_text_concat</th>\n",
       "      <th>chunk_title_text_concat</th>\n",
       "      <th>prompt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MED-3931</td>\n",
       "      <td>Pilot dietary study with normoproteic protein-...</td>\n",
       "      <td>Although a plant-based diet can provide some b...</td>\n",
       "      <td>2386</td>\n",
       "      <td>Although a plant-based diet can provide some b...</td>\n",
       "      <td>0</td>\n",
       "      <td>Pilot dietary study with normoproteic protein-...</td>\n",
       "      <td>Pilot dietary study with normoproteic protein-...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MED-4147</td>\n",
       "      <td>Occurrence of steroid hormones and antibiotics...</td>\n",
       "      <td>Wastewater impoundments at concentrated animal...</td>\n",
       "      <td>2546</td>\n",
       "      <td>Wastewater impoundments at concentrated animal...</td>\n",
       "      <td>0</td>\n",
       "      <td>Occurrence of steroid hormones and antibiotics...</td>\n",
       "      <td>Occurrence of steroid hormones and antibiotics...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MED-2335</td>\n",
       "      <td>Xenohormesis: health benefits from an eon of p...</td>\n",
       "      <td>Xenohormesis is a biological principle that ex...</td>\n",
       "      <td>1296</td>\n",
       "      <td>Xenohormesis is a biological principle that ex...</td>\n",
       "      <td>0</td>\n",
       "      <td>Xenohormesis: health benefits from an eon of p...</td>\n",
       "      <td>Xenohormesis: health benefits from an eon of p...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MED-5120</td>\n",
       "      <td>L-theanine intervention enhances human gammade...</td>\n",
       "      <td>Human gammadeltaT lymphocytes are a subset of ...</td>\n",
       "      <td>3369</td>\n",
       "      <td>Human gammadeltaT lymphocytes are a subset of ...</td>\n",
       "      <td>0</td>\n",
       "      <td>L-theanine intervention enhances human gammade...</td>\n",
       "      <td>L-theanine intervention enhances human gammade...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MED-1002</td>\n",
       "      <td>Polybrominated diphenyl ethers (PBDEs), hydrox...</td>\n",
       "      <td>Prenatal exposure to polybrominated diphenyl e...</td>\n",
       "      <td>202</td>\n",
       "      <td>Prenatal exposure to polybrominated diphenyl e...</td>\n",
       "      <td>0</td>\n",
       "      <td>Polybrominated diphenyl ethers (PBDEs), hydrox...</td>\n",
       "      <td>Polybrominated diphenyl ethers (PBDEs), hydrox...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>MED-4462</td>\n",
       "      <td>Sulforaphane protects human chondrocytes again...</td>\n",
       "      <td>Chondrocyte cell death can contribute to carti...</td>\n",
       "      <td>2787</td>\n",
       "      <td>Chondrocyte cell death can contribute to carti...</td>\n",
       "      <td>0</td>\n",
       "      <td>Sulforaphane protects human chondrocytes again...</td>\n",
       "      <td>Sulforaphane protects human chondrocytes again...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>MED-4007</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>2450</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>0</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>MED-4007</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Background Alzheimer's disease (AD) is charact...</td>\n",
       "      <td>2450</td>\n",
       "      <td>In the per protocol population, E4(-) particip...</td>\n",
       "      <td>1</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>Study of the ketogenic agent AC-1202 in mild t...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>MED-4550</td>\n",
       "      <td>An impact of the diet on serum fatty acid and ...</td>\n",
       "      <td>BACKGROUND/OBJECTIVES: Vegetarian diet has bec...</td>\n",
       "      <td>2862</td>\n",
       "      <td>BACKGROUND/OBJECTIVES: Vegetarian diet has bec...</td>\n",
       "      <td>0</td>\n",
       "      <td>An impact of the diet on serum fatty acid and ...</td>\n",
       "      <td>An impact of the diet on serum fatty acid and ...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>MED-4361</td>\n",
       "      <td>Dirty money: an investigation into the hygiene...</td>\n",
       "      <td>A total of 1280 banknotes were obtained from f...</td>\n",
       "      <td>2703</td>\n",
       "      <td>A total of 1280 banknotes were obtained from f...</td>\n",
       "      <td>0</td>\n",
       "      <td>Dirty money: an investigation into the hygiene...</td>\n",
       "      <td>Dirty money: an investigation into the hygiene...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>101 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          _id                                              title  \\\n",
       "0    MED-3931  Pilot dietary study with normoproteic protein-...   \n",
       "1    MED-4147  Occurrence of steroid hormones and antibiotics...   \n",
       "2    MED-2335  Xenohormesis: health benefits from an eon of p...   \n",
       "3    MED-5120  L-theanine intervention enhances human gammade...   \n",
       "4    MED-1002  Polybrominated diphenyl ethers (PBDEs), hydrox...   \n",
       "..        ...                                                ...   \n",
       "96   MED-4462  Sulforaphane protects human chondrocytes again...   \n",
       "97   MED-4007  Study of the ketogenic agent AC-1202 in mild t...   \n",
       "98   MED-4007  Study of the ketogenic agent AC-1202 in mild t...   \n",
       "99   MED-4550  An impact of the diet on serum fatty acid and ...   \n",
       "100  MED-4361  Dirty money: an investigation into the hygiene...   \n",
       "\n",
       "                                                  text  paragraph_id  \\\n",
       "0    Although a plant-based diet can provide some b...          2386   \n",
       "1    Wastewater impoundments at concentrated animal...          2546   \n",
       "2    Xenohormesis is a biological principle that ex...          1296   \n",
       "3    Human gammadeltaT lymphocytes are a subset of ...          3369   \n",
       "4    Prenatal exposure to polybrominated diphenyl e...           202   \n",
       "..                                                 ...           ...   \n",
       "96   Chondrocyte cell death can contribute to carti...          2787   \n",
       "97   Background Alzheimer's disease (AD) is charact...          2450   \n",
       "98   Background Alzheimer's disease (AD) is charact...          2450   \n",
       "99   BACKGROUND/OBJECTIVES: Vegetarian diet has bec...          2862   \n",
       "100  A total of 1280 banknotes were obtained from f...          2703   \n",
       "\n",
       "                                            chunk_text  chunk_id  \\\n",
       "0    Although a plant-based diet can provide some b...         0   \n",
       "1    Wastewater impoundments at concentrated animal...         0   \n",
       "2    Xenohormesis is a biological principle that ex...         0   \n",
       "3    Human gammadeltaT lymphocytes are a subset of ...         0   \n",
       "4    Prenatal exposure to polybrominated diphenyl e...         0   \n",
       "..                                                 ...       ...   \n",
       "96   Chondrocyte cell death can contribute to carti...         0   \n",
       "97   Background Alzheimer's disease (AD) is charact...         0   \n",
       "98   In the per protocol population, E4(-) particip...         1   \n",
       "99   BACKGROUND/OBJECTIVES: Vegetarian diet has bec...         0   \n",
       "100  A total of 1280 banknotes were obtained from f...         0   \n",
       "\n",
       "                                     title_text_concat  \\\n",
       "0    Pilot dietary study with normoproteic protein-...   \n",
       "1    Occurrence of steroid hormones and antibiotics...   \n",
       "2    Xenohormesis: health benefits from an eon of p...   \n",
       "3    L-theanine intervention enhances human gammade...   \n",
       "4    Polybrominated diphenyl ethers (PBDEs), hydrox...   \n",
       "..                                                 ...   \n",
       "96   Sulforaphane protects human chondrocytes again...   \n",
       "97   Study of the ketogenic agent AC-1202 in mild t...   \n",
       "98   Study of the ketogenic agent AC-1202 in mild t...   \n",
       "99   An impact of the diet on serum fatty acid and ...   \n",
       "100  Dirty money: an investigation into the hygiene...   \n",
       "\n",
       "                               chunk_title_text_concat  \\\n",
       "0    Pilot dietary study with normoproteic protein-...   \n",
       "1    Occurrence of steroid hormones and antibiotics...   \n",
       "2    Xenohormesis: health benefits from an eon of p...   \n",
       "3    L-theanine intervention enhances human gammade...   \n",
       "4    Polybrominated diphenyl ethers (PBDEs), hydrox...   \n",
       "..                                                 ...   \n",
       "96   Sulforaphane protects human chondrocytes again...   \n",
       "97   Study of the ketogenic agent AC-1202 in mild t...   \n",
       "98   Study of the ketogenic agent AC-1202 in mild t...   \n",
       "99   An impact of the diet on serum fatty acid and ...   \n",
       "100  Dirty money: an investigation into the hygiene...   \n",
       "\n",
       "                                                prompt  \n",
       "0    <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "1    <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "2    <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "3    <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "4    <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "..                                                 ...  \n",
       "96   <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "97   <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "98   <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "99   <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "100  <s>[INST] <<SYS>>\\nYou are a data annotator tr...  \n",
       "\n",
       "[101 rows x 9 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kb[\"prompt\"] = kb[\"chunk_title_text_concat\"].apply(get_prompt, system_prompt=system_prompt, chat_history=\"\")\n",
    "kb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Synthetic Data Generation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the function used to generate examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def generate_multiple(texts, ):\n",
    "    port_num = 5555\n",
    "    headers = {\"Content-Type\": \"application/json\"}\n",
    "    if isinstance(texts, str):\n",
    "        prompts = [texts]\n",
    "    elif isinstance(texts, list):\n",
    "        prompts = texts\n",
    "    data = {\n",
    "        \"sentences\": prompts,\n",
    "        \"tokens_to_generate\": TOKENS_TO_GENERATE,\n",
    "        \"temperature\": TEMPERATURE,\n",
    "        \"add_BOS\": True,\n",
    "        \"top_k\": TOP_K,\n",
    "        \"top_p\": TOP_P,\n",
    "        \"greedy\": False,\n",
    "        \"all_probs\": False,\n",
    "        \"repetition_penalty\": 1.2,\n",
    "        \"min_tokens_to_generate\": 2,\n",
    "        }\n",
    "    resp = requests.put('http://localhost:{}/generate'.format(port_num),\n",
    "                        data=json.dumps(data),\n",
    "                        headers=headers)\n",
    "    return resp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "batch_size= 4\n",
    "texts = kb[\"prompt\"].tolist()\n",
    "len_texts = len(texts)\n",
    "generations = []\n",
    "for i in range(0, len_texts, batch_size):\n",
    "    st_time = time.time()\n",
    "    if i + batch_size <= len_texts:\n",
    "        resp = generate_multiple(texts[i: i + batch_size])\n",
    "        print (f'Completed batch --> {i}:{i+batch_size}')\n",
    "    else:\n",
    "        resp = generate_multiple(texts[i: len_texts])\n",
    "        print (f'Completed batch --> {i}:{len_texts}')\n",
    "        \n",
    "    generations.extend(resp.json()['sentences'])\n",
    "    print ('Time taken per batch = {:.2f} s'.format(time.time()-st_time))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "kb[\"generated_text\"] = generations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parsing generations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Parse the generations to extract the generate queries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def extract_questions_from_generations(kb):\n",
    "    paragraph_id_question = []\n",
    "    for row in kb.to_dict(orient='records'):\n",
    "        paragraph_id = row[\"paragraph_id\"]\n",
    "        title = row[\"title\"]\n",
    "        text = row[\"chunk_text\"]\n",
    "        chunk_id = row[\"chunk_id\"]\n",
    "        print(row[\"generated_text\"])\n",
    "        questions = re.findall(r'<q>(.+?)</q>', row[\"generated_text\"])\n",
    "        print(questions)\n",
    "        print(\"-\"*200)\n",
    "        paragraph_id_question += [(paragraph_id, chunk_id, question) for question in questions]\n",
    "    return pd.DataFrame(paragraph_id_question, columns=[\"paragraph_id\", \"chunk_id\", \"chunk_question\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the health risks associated with consuming beverages sweetened with high-fructose corn syrup?</q>\n",
      "2. <q>How does fructose intake from sugar-sweetened beverages affect liver function and metabolism?</q>\n",
      "3. <q>What are the differences in metabolic effects between glucose and fructose in sugar-sweetened beverages?</q>\n",
      "\n",
      "These queries are based on the information in Document 2 regarding the health risks associated with high-fructose corn syrup and the metabolic effects of fructose and glucose in sugar-sweetened beverages.\n",
      "['What are the health risks associated with consuming beverages sweetened with high-fructose corn syrup?', 'How does fructose intake from sugar-sweetened beverages affect liver function and metabolism?', 'What are the differences in metabolic effects between glucose and fructose in sugar-sweetened beverages?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the association between hot dog consumption and childhood leukemia risk?</q>\n",
      "2. <q>Do processed meats increase the risk of leukemia in children?</q>\n",
      "3. <q>Can fruit consumption reduce the risk of leukemia in children?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not rely on the text from Example.\n",
      "['What is the association between hot dog consumption and childhood leukemia risk?', 'Do processed meats increase the risk of leukemia in children?', 'Can fruit consumption reduce the risk of leukemia in children?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the health benefits of adhering to a healthy lifestyle?</q>\n",
      "2. <q>How can I reduce my risk of developing chronic diseases like diabetes and cancer?</q>\n",
      "3. <q>What is the relationship between a healthy diet and the prevention of chronic diseases?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the reduction in relative risk of developing major chronic diseases associated with four healthy lifestyle factors, such as never smoking, having a healthy body mass index, performing regular physical activity, and adhering to healthy dietary principles.\n",
      "['What are the health benefits of adhering to a healthy lifestyle?', 'How can I reduce my risk of developing chronic diseases like diabetes and cancer?', 'What is the relationship between a healthy diet and the prevention of chronic diseases?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the potential sources of arsenic in organic food products?</q>\n",
      "2. <q>How does the use of organic brown rice syrup in food products affect the concentration of arsenic in these products?</q>\n",
      "3. <q>Are there any regulatory limits on the amount of arsenic in food products, and if not, what are the potential health risks associated with consuming high levels of arsenic in food?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the presence of arsenic in organic brown rice syrup and its potential impact on food products. The queries are designed to elicit information about the sources of arsenic in food, the effects of organic brown rice syrup on arsenic levels, and the potential health risks associated with consuming high levels of arsenic in food.\n",
      "['What are the potential sources of arsenic in organic food products?', 'How does the use of organic brown rice syrup in food products affect the concentration of arsenic in these products?', 'Are there any regulatory limits on the amount of arsenic in food products, and if not, what are the potential health risks associated with consuming high levels of arsenic in food?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How does curcumin sensitize drug-resistant tumors?</q>\n",
      "2. <q>What are the mechanisms of drug resistance in cancer, and how can curcumin overcome them?</q>\n",
      "3. <q>Can curcumin target cancer stem cells to circumvent drug resistance in tumors?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, and do not use any text from the example provided.\n",
      "['How does curcumin sensitize drug-resistant tumors?', 'What are the mechanisms of drug resistance in cancer, and how can curcumin overcome them?', 'Can curcumin target cancer stem cells to circumvent drug resistance in tumors?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the best sources of antioxidants for protecting neuronal cells?</q>\n",
      "2. <q>How do the phenolic content and antineurodegenerative activities of sweet and sour cherries compare?</q>\n",
      "3. <q>What is the most effective method for measuring the anthocyanin content of cherries?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, but do not directly copy any text from the document.\n",
      "['What are the best sources of antioxidants for protecting neuronal cells?', 'How do the phenolic content and antineurodegenerative activities of sweet and sour cherries compare?', 'What is the most effective method for measuring the anthocyanin content of cherries?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the effects of sulfide on colonic mucosa proliferation?</q>\n",
      "2. <q>How does butyrate regulate colonic proliferation in the treatment of ulcerative colitis?</q>\n",
      "3. <q>What is the role of sulfide in the pathogenesis of ulcerative colitis?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the antagonistic effects of sulfide and butyrate on colonic mucosa proliferation, and the potential role of these agents in the pathogenesis of ulcerative colitis.\n",
      "['What are the effects of sulfide on colonic mucosa proliferation?', 'How does butyrate regulate colonic proliferation in the treatment of ulcerative colitis?', 'What is the role of sulfide in the pathogenesis of ulcerative colitis?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the cause of hepatitis induced by Noni juice from Morinda citrifolia?</q>\n",
      "2. <q>Can interferon beta-1a withdrawal lead to fulminant hepatitis and acute liver failure?</q>\n",
      "3. <q>What is the relationship between drinking Noni juice and drug-induced hepatitis?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What is the cause of hepatitis induced by Noni juice from Morinda citrifolia?', 'Can interferon beta-1a withdrawal lead to fulminant hepatitis and acute liver failure?', 'What is the relationship between drinking Noni juice and drug-induced hepatitis?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the effect of adenovirus infection on chicken adiposity?</q>\n",
      "2. <q>How does adenovirus infection affect chicken weight gain?</q>\n",
      "3. <q>What are the differences in serum cholesterol and triglyceride levels between adenovirus-infected and non-infected chickens?</q>\n",
      "['What is the effect of adenovirus infection on chicken adiposity?', 'How does adenovirus infection affect chicken weight gain?', 'What are the differences in serum cholesterol and triglyceride levels between adenovirus-infected and non-infected chickens?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the relationship between adherence to a Mediterranean diet and the risk of cognitive impairment in older adults?</q>\n",
      "2. <q>Can a Mediterranean diet help prevent depression in middle-aged and older adults?</q>\n",
      "3. <q>How does adherence to a Mediterranean diet affect the risk of stroke in males versus females?</q>\n",
      "\n",
      "These queries are based on the key points of Document 2, such as the association between Mediterranean diet and brain diseases, the protective effects of high adherence, and the potential differences in effects based on gender.\n",
      "['What is the relationship between adherence to a Mediterranean diet and the risk of cognitive impairment in older adults?', 'Can a Mediterranean diet help prevent depression in middle-aged and older adults?', 'How does adherence to a Mediterranean diet affect the risk of stroke in males versus females?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the best breakfast options to reduce inflammation in the body?</q>\n",
      "2. <q>How does the type of fat consumed in breakfast affect the expression of proinflammatory genes in PBMCs?</q>\n",
      "3. <q>Can a walnut-enriched breakfast help reduce the postprandial inflammatory response in healthy men?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and are not simply copies of text from the document.\n",
      "['What are the best breakfast options to reduce inflammation in the body?', 'How does the type of fat consumed in breakfast affect the expression of proinflammatory genes in PBMCs?', 'Can a walnut-enriched breakfast help reduce the postprandial inflammatory response in healthy men?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the projected prevalence of obesity in the US by 2030?</q>\n",
      "2. <q>How do annual increase rates of obesity differ between population groups in the US?</q>\n",
      "3. <q>What are the estimated healthcare costs attributable to obesity/overweight in the US by 2030?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What is the projected prevalence of obesity in the US by 2030?', 'How do annual increase rates of obesity differ between population groups in the US?', 'What are the estimated healthcare costs attributable to obesity/overweight in the US by 2030?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the effects of high-intensity exercise on lipid peroxidation in the human body?</q>\n",
      "2. <q>How does exercise-induced DNA damage relate to the production of free radicals during aerobic exercise?</q>\n",
      "3. <q>What are the implications of lipid-derived free radicals on immune function and the aging process?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What are the effects of high-intensity exercise on lipid peroxidation in the human body?', 'How does exercise-induced DNA damage relate to the production of free radicals during aerobic exercise?', 'What are the implications of lipid-derived free radicals on immune function and the aging process?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the lifestyle changes I can make to prevent cancer?</q>\n",
      "2. <q>How does smoking affect my risk of developing cancer?</q>\n",
      "3. <q>What is the link between inflammation and cancer prevention?</q>\n",
      "\n",
      "These queries are based on the information presented in Document 2, which highlights the importance of lifestyle changes in preventing cancer, the dangers of smoking, and the role of inflammation in cancer prevention.\n",
      "['What are the lifestyle changes I can make to prevent cancer?', 'How does smoking affect my risk of developing cancer?', 'What is the link between inflammation and cancer prevention?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the association between heme iron intake and the risk of developing type 2 diabetes in a Mediterranean population?</q>\n",
      "2. <q>How does coffee consumption affect the risk of developing type 2 diabetes in a high-risk population?</q>\n",
      "3. <q>What is the relationship between dietary heme iron intake and the incidence of type 2 diabetes in a population at high cardiovascular risk?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, specifically focusing on the association between heme iron intake and the risk of developing type 2 diabetes, as well as the potential modifying effects of coffee and other dietary factors.\n",
      "['What is the association between heme iron intake and the risk of developing type 2 diabetes in a Mediterranean population?', 'How does coffee consumption affect the risk of developing type 2 diabetes in a high-risk population?', 'What is the relationship between dietary heme iron intake and the incidence of type 2 diabetes in a population at high cardiovascular risk?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the effects of a vegetarian diet on plasma phospholipid fatty acids in rheumatoid arthritis patients?</q>\n",
      "2. <q>Can manipulation of dietary fatty acids improve disease activity in rheumatoid arthritis patients?</q>\n",
      "3. <q>What is the relationship between changes in plasma phospholipid fatty acids and clinical improvement in rheumatoid arthritis patients treated with a vegetarian diet?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the changes in plasma phospholipid fatty acids in rheumatoid arthritis patients treated with a vegetarian diet, and the relationship between these changes and disease activity.\n",
      "['What are the effects of a vegetarian diet on plasma phospholipid fatty acids in rheumatoid arthritis patients?', 'Can manipulation of dietary fatty acids improve disease activity in rheumatoid arthritis patients?', 'What is the relationship between changes in plasma phospholipid fatty acids and clinical improvement in rheumatoid arthritis patients treated with a vegetarian diet?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the risk factors for symptomatic disc herniation?</q>\n",
      "2. <q>How do serum lipid levels affect intervertebral disc degeneration?</q>\n",
      "3. <q>Can poor blood supply to the intervertebral disc increase the risk of ischaemic heart disease?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the association between intervertebral disc degeneration and risk factors such as obesity, tobacco use, and serum lipid levels. The queries do not rely on the text of Example, and are instead based on the specific information presented in Document 2.\n",
      "['What are the risk factors for symptomatic disc herniation?', 'How do serum lipid levels affect intervertebral disc degeneration?', 'Can poor blood supply to the intervertebral disc increase the risk of ischaemic heart disease?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the lifestyle risk factors for abdominal aortic aneurysm?</q>\n",
      "2. <q>How does smoking cessation affect the risk of developing abdominal aortic aneurysm?</q>\n",
      "3. <q>What is the predictive scoring system for identifying abdominal aortic aneurysms in high-risk individuals?</q>\n",
      "\n",
      "These queries are based on the information in Document 2, which discusses the risk factors for abdominal aortic aneurysm, the association between smoking and AAA risk, and the development of a predictive scoring system for identifying high-risk individuals.\n",
      "['What are the lifestyle risk factors for abdominal aortic aneurysm?', 'How does smoking cessation affect the risk of developing abdominal aortic aneurysm?', 'What is the predictive scoring system for identifying abdominal aortic aneurysms in high-risk individuals?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the association between milk fat consumption and testicular cancer risk in adolescence?</q>\n",
      "2. <q>How does galactose consumption affect the risk of testicular cancer in adolescence?</q>\n",
      "3. <q>Can the risk of seminomatous testicular cancer be explained by milk fat and/or galactose consumption in adolescence?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the association between dairy product consumption, specifically milk, milk fat, and galactose, and testicular cancer in adolescence. The queries are designed to elicit information about the relationship between these factors and the risk of testicular cancer.\n",
      "['What is the association between milk fat consumption and testicular cancer risk in adolescence?', 'How does galactose consumption affect the risk of testicular cancer in adolescence?', 'Can the risk of seminomatous testicular cancer be explained by milk fat and/or galactose consumption in adolescence?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the recommended dosage of cisapride for the treatment of gastrointestinal disorders?</q>\n",
      "2. <q>Can cisapride be used to treat chronic intestinal pseudo-obstruction syndrome?</q>\n",
      "3. <q>What are the long-term effects of cisapride on gastric emptying rates in patients with gastroparesis?</q>\n",
      "\n",
      "Note that these queries are based on the information provided in Document 2, and do not rely on the information from Example.\n",
      "['What is the recommended dosage of cisapride for the treatment of gastrointestinal disorders?', 'Can cisapride be used to treat chronic intestinal pseudo-obstruction syndrome?', 'What are the long-term effects of cisapride on gastric emptying rates in patients with gastroparesis?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the relationship between anthroposophic lifestyle and gut flora in infancy?</q>\n",
      "2. <q>How does the composition of gut flora differ between infants with an anthroposophic lifestyle and those with a traditional lifestyle?</q>\n",
      "3. <q>What are the specific lifestyle features that influence the diversity of lactobacilli in the gut flora of infants?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What is the relationship between anthroposophic lifestyle and gut flora in infancy?', 'How does the composition of gut flora differ between infants with an anthroposophic lifestyle and those with a traditional lifestyle?', 'What are the specific lifestyle features that influence the diversity of lactobacilli in the gut flora of infants?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How can I identify if I have non-responsive celiac disease and what are the possible causes?</q>\n",
      "2. <q>What are the common mistakes in following a gluten-free diet that can lead to persistent symptoms in celiac disease patients?</q>\n",
      "3. <q>What are the alternative diagnoses that should be considered in patients with non-responsive celiac disease, and how are they diagnosed?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, but do not directly copy any text from the document.\n",
      "['How can I identify if I have non-responsive celiac disease and what are the possible causes?', 'What are the common mistakes in following a gluten-free diet that can lead to persistent symptoms in celiac disease patients?', 'What are the alternative diagnoses that should be considered in patients with non-responsive celiac disease, and how are they diagnosed?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated from Document 2:\n",
      "\n",
      "1. <q>What is the association between severe constipation and cytological abnormalities in breast fluid?</q>\n",
      "2. <q>How does the frequency of bowel movements affect the risk of epithelial dysplasia in nipple aspirates of breast fluid?</q>\n",
      "3. <q>What is the potential link between diet and breast disease in women with severe constipation, and how may intestinal flora play a role in this association?</q>\n",
      "['What is the association between severe constipation and cytological abnormalities in breast fluid?', 'How does the frequency of bowel movements affect the risk of epithelial dysplasia in nipple aspirates of breast fluid?', 'What is the potential link between diet and breast disease in women with severe constipation, and how may intestinal flora play a role in this association?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the recommended haemoglobin level for pregnant women to avoid low birth weight and preterm birth?</q>\n",
      "2. <q>Can pregnant women with high haemoglobin levels during pregnancy still benefit from iron supplements?</q>\n",
      "3. <q>What is the safe range of haemoglobin levels for pregnant women during the 1st, 2nd, and 3rd trimesters?</q>\n",
      "\n",
      "Note that these queries are based on the information provided in Document 2, and do not rely on the information from Example.\n",
      "['What is the recommended haemoglobin level for pregnant women to avoid low birth weight and preterm birth?', 'Can pregnant women with high haemoglobin levels during pregnancy still benefit from iron supplements?', 'What is the safe range of haemoglobin levels for pregnant women during the 1st, 2nd, and 3rd trimesters?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the potential toxic effects of protein fermentation on gut health?</q>\n",
      "2. <q>How does the fermentation of proteins in the distal colon affect the development of colorectal cancer?</q>\n",
      "3. <q>Can a high-meat diet increase the risk of colorectal cancer due to protein fermentation?</q>\n",
      "\n",
      "These queries are based on the information in Document 2, which suggests that protein fermentation can have detrimental effects on gut health, particularly in the distal colon, and that a high-meat diet may increase the risk of colorectal cancer due to the increased fermentation of proteins.\n",
      "['What are the potential toxic effects of protein fermentation on gut health?', 'How does the fermentation of proteins in the distal colon affect the development of colorectal cancer?', 'Can a high-meat diet increase the risk of colorectal cancer due to protein fermentation?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the average percentage of natural killer cell activity in highly trained individuals?</q>\n",
      "2. <q>How does indomethacin affect natural killer cell activity in peripheral blood?</q>\n",
      "3. <q>What is the difference in CD16+ cell concentration between highly trained and untrained individuals?</q>\n",
      "\n",
      "Note that these queries are based on the information provided in Document 2, and do not rely on the information from Example.\n",
      "['What is the average percentage of natural killer cell activity in highly trained individuals?', 'How does indomethacin affect natural killer cell activity in peripheral blood?', 'What is the difference in CD16+ cell concentration between highly trained and untrained individuals?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the average serum total cholesterol level among adult Yanomamo Indians in Brazil?</q>\n",
      "2. <q>How does the diet and lifestyle of Yanomamo Indians in Brazil compare to western populations in terms of lipid profiles?</q>\n",
      "3. <q>What is the relationship between body mass index (BMI) and serum lipid levels among Yanomamo Indians in Brazil?</q>\n",
      "\n",
      "Note that these queries are based on the information provided in Document 2, and do not rely on the text from Example.\n",
      "['What is the average serum total cholesterol level among adult Yanomamo Indians in Brazil?', 'How does the diet and lifestyle of Yanomamo Indians in Brazil compare to western populations in terms of lipid profiles?', 'What is the relationship between body mass index (BMI) and serum lipid levels among Yanomamo Indians in Brazil?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the microvascular and macrovascular complications of type 2 diabetes mellitus in children and adolescents?</q>\n",
      "2. <q>How does obesity contribute to the development of type 2 diabetes mellitus in children and adolescents?</q>\n",
      "3. <q>What is the appropriate screening process for type 2 diabetes mellitus in children and adolescents at risk for the disease?</q>\n",
      "['What are the microvascular and macrovascular complications of type 2 diabetes mellitus in children and adolescents?', 'How does obesity contribute to the development of type 2 diabetes mellitus in children and adolescents?', 'What is the appropriate screening process for type 2 diabetes mellitus in children and adolescents at risk for the disease?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the association between dairy protein intake and prostate cancer risk?</q>\n",
      "2. <q>How does calcium from dairy products affect prostate cancer risk, compared to calcium from other sources?</q>\n",
      "3. <q>What is the relationship between animal food consumption and prostate cancer risk, and how does this vary by stage of disease?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, but do not directly copy any text from the document.\n",
      "['What is the association between dairy protein intake and prostate cancer risk?', 'How does calcium from dairy products affect prostate cancer risk, compared to calcium from other sources?', 'What is the relationship between animal food consumption and prostate cancer risk, and how does this vary by stage of disease?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the risk factors for developing erectile dysfunction (ED) later in life?</q>\n",
      "2. <q>Can improving coronary heart disease (CHD) risk factors in mid-life reduce the risk of developing ED?</q>\n",
      "3. <q>Are there any specific lifestyle modifications or medications that can help prevent or treat ED?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the risk factors for erectile dysfunction (ED) and the potential benefits of improving coronary heart disease (CHD) risk factors in mid-life for reducing the risk of ED.\n",
      "['What are the risk factors for developing erectile dysfunction (ED) later in life?', 'Can improving coronary heart disease (CHD) risk factors in mid-life reduce the risk of developing ED?', 'Are there any specific lifestyle modifications or medications that can help prevent or treat ED?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How can I incorporate mindfulness training into my dietary change program for prostate cancer?</q>\n",
      "2. <q>What is the optimal animal:vegetable protein ratio for a prostate-healthy diet?</q>\n",
      "3. <q>How can I sustain dietary changes over time for prostate cancer prevention and management?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the use of a ratio of animal to vegetable proteins to evaluate dietary change, the incorporation of mindfulness training into a dietary program, and the importance of sustaining dietary changes over time for prostate cancer prevention and management.\n",
      "['How can I incorporate mindfulness training into my dietary change program for prostate cancer?', 'What is the optimal animal:vegetable protein ratio for a prostate-healthy diet?', 'How can I sustain dietary changes over time for prostate cancer prevention and management?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the recommended dosage of acai (Euterpe oleracea) for weight loss?</q>\n",
      "2. <q>Does acai (Euterpe oleracea) interact with any medications or have adverse effects?</q>\n",
      "3. <q>What is the evidence for the efficacy of acai (Euterpe oleracea) in reducing inflammation?</q>\n",
      "\n",
      "These queries are based on the topics covered in Document 2, such as dosing, interactions, adverse effects, and efficacy. They are designed to be answerable based on the information provided in the document, and can help users find relevant information within the text.\n",
      "['What is the recommended dosage of acai (Euterpe oleracea) for weight loss?', 'Does acai (Euterpe oleracea) interact with any medications or have adverse effects?', 'What is the evidence for the efficacy of acai (Euterpe oleracea) in reducing inflammation?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Based on the content of Document 2, the following are three search queries that can be generated:\n",
      "\n",
      "1. <q>What is the relationship between diverticulosis and dietary fiber intake?</q>\n",
      "2. <q>Can constipation increase the risk of developing diverticulosis?</q>\n",
      "3. <q>What are the factors that contribute to the development of diverticulosis?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What is the relationship between diverticulosis and dietary fiber intake?', 'Can constipation increase the risk of developing diverticulosis?', 'What are the factors that contribute to the development of diverticulosis?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the effects of caffeine on vascular function in humans?</q>\n",
      "2. <q>How does caffeine affect endothelial function in healthy young men?</q>\n",
      "3. <q>What is the mechanism by which caffeine augments endothelium-dependent vasodilation?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not rely on the text from Example.\n",
      "['What are the effects of caffeine on vascular function in humans?', 'How does caffeine affect endothelial function in healthy young men?', 'What is the mechanism by which caffeine augments endothelium-dependent vasodilation?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the relationship between plasma amino acid levels and motor fluctuations in Parkinson's disease patients?</q>\n",
      "2. <q>How does a low-protein diet affect the symptoms of Parkinson's disease patients who experience motor fluctuations?</q>\n",
      "3. <q>What is the effect of redistribution diet on motor function in Parkinson's disease patients with severe motor fluctuations?</q>\n",
      "[\"What is the relationship between plasma amino acid levels and motor fluctuations in Parkinson's disease patients?\", \"How does a low-protein diet affect the symptoms of Parkinson's disease patients who experience motor fluctuations?\", \"What is the effect of redistribution diet on motor function in Parkinson's disease patients with severe motor fluctuations?\"]\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the risk factors for predicting postoperative nausea and vomiting in adult patients undergoing inhalational anesthesia for various types of surgery?</q>\n",
      "2. <q>How can I simplify the risk score for predicting postoperative nausea and vomiting without losing its discriminating power?</q>\n",
      "3. <q>What is the validity of a risk score derived from one center in predicting postoperative nausea and vomiting in patients from another center?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What are the risk factors for predicting postoperative nausea and vomiting in adult patients undergoing inhalational anesthesia for various types of surgery?', 'How can I simplify the risk score for predicting postoperative nausea and vomiting without losing its discriminating power?', 'What is the validity of a risk score derived from one center in predicting postoperative nausea and vomiting in patients from another center?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the effects of cocoa powder on plasma LDL cholesterol and oxidized LDL concentrations in humans?</q>\n",
      "2. <q>How does the intake of different levels of cocoa powder affect plasma HDL cholesterol concentrations in normocholesterolemic and mildly hypercholesterolemic humans?</q>\n",
      "3. <q>What are the potential benefits of polyphenolic substances derived from cocoa powder on cardiovascular health, specifically in reducing LDL cholesterol and oxidized LDL, and increasing HDL cholesterol?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What are the effects of cocoa powder on plasma LDL cholesterol and oxidized LDL concentrations in humans?', 'How does the intake of different levels of cocoa powder affect plasma HDL cholesterol concentrations in normocholesterolemic and mildly hypercholesterolemic humans?', 'What are the potential benefits of polyphenolic substances derived from cocoa powder on cardiovascular health, specifically in reducing LDL cholesterol and oxidized LDL, and increasing HDL cholesterol?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the potential carcinogenic compounds formed in processed meat during cooking?</q>\n",
      "2. <q>How does the intake of processed meat affect the risk of colorectal cancer, and what are the epidemiologic and experimental evidence supporting this association?</q>\n",
      "3. <q>What are the possible mechanisms by which processed meat intake may increase the risk of colorectal cancer, and how can these be tested experimentally?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, and do not use any text from the Example provided.\n",
      "['What are the potential carcinogenic compounds formed in processed meat during cooking?', 'How does the intake of processed meat affect the risk of colorectal cancer, and what are the epidemiologic and experimental evidence supporting this association?', 'What are the possible mechanisms by which processed meat intake may increase the risk of colorectal cancer, and how can these be tested experimentally?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the association between green tea consumption and breast cancer recurrence risk?</q>\n",
      "2. <q>Can green tea consumption reduce the risk of breast cancer incidence?</q>\n",
      "3. <q>What is the optimal amount of green tea consumption for breast cancer prevention?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and are not based on the text of Example.\n",
      "['What is the association between green tea consumption and breast cancer recurrence risk?', 'Can green tea consumption reduce the risk of breast cancer incidence?', 'What is the optimal amount of green tea consumption for breast cancer prevention?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the most effective treatment for quiescent ulcerative colitis?</q>\n",
      "2. <q>Can curcumin therapy help suppress the morbidity associated with ulcerative colitis?</q>\n",
      "3. <q>What is the recommended dosage of curcumin for maintenance therapy in ulcerative colitis patients?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, but do not directly copy any text from the document.\n",
      "['What is the most effective treatment for quiescent ulcerative colitis?', 'Can curcumin therapy help suppress the morbidity associated with ulcerative colitis?', 'What is the recommended dosage of curcumin for maintenance therapy in ulcerative colitis patients?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the chemical safety concerns associated with the consumption of meat and meat products?</q>\n",
      "2. <q>How can the use of nitrite in meat products affect consumer health and safety?</q>\n",
      "3. <q>What is the Failure Mode and Effect Analysis (FMEA) system and how is it used to assess vulnerabilities in the food chain?</q>\n",
      "\n",
      "Note that these queries are not based on the text of Example, but rather on the content of Document 2.\n",
      "['What are the chemical safety concerns associated with the consumption of meat and meat products?', 'How can the use of nitrite in meat products affect consumer health and safety?', 'What is the Failure Mode and Effect Analysis (FMEA) system and how is it used to assess vulnerabilities in the food chain?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the dietary sources of monoamine oxidase A inhibitors?</q>\n",
      "2. <q>How does the pharmacokinetics of flavonoids, such as quercetin, affect their ability to inhibit monoamine oxidase A?</q>\n",
      "3. <q>What is the estimated daily intake of quercetin needed to achieve significant levels of monoamine oxidase A inhibition in the brain?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from Example.\n",
      "['What are the dietary sources of monoamine oxidase A inhibitors?', 'How does the pharmacokinetics of flavonoids, such as quercetin, affect their ability to inhibit monoamine oxidase A?', 'What is the estimated daily intake of quercetin needed to achieve significant levels of monoamine oxidase A inhibition in the brain?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How does overcrowding stress affect the immune system of broiler chickens?</q>\n",
      "2. <q>What are the effects of overcrowding stress on the intestinal health of broiler chickens?</q>\n",
      "3. <q>Can overcrowding stress increase the susceptibility of broiler chickens to Salmonella infection?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['How does overcrowding stress affect the immune system of broiler chickens?', 'What are the effects of overcrowding stress on the intestinal health of broiler chickens?', 'Can overcrowding stress increase the susceptibility of broiler chickens to Salmonella infection?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the correlation between tick bites and the production of IgE antibodies to galactose-α-1,3-galactose?</q>\n",
      "2. <q>How do tick bites contribute to the development of food allergy to red meat?</q>\n",
      "3. <q>What is the significance of the correlation between IgE antibodies to alpha-gal and IgE antibodies to proteins derived from the tick Amblyomma americanum?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What is the correlation between tick bites and the production of IgE antibodies to galactose-α-1,3-galactose?', 'How do tick bites contribute to the development of food allergy to red meat?', 'What is the significance of the correlation between IgE antibodies to alpha-gal and IgE antibodies to proteins derived from the tick Amblyomma americanum?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the nutritional differences between organic and non-organic crops?</q>\n",
      "2. <q>How do organic farming practices affect the concentration of antioxidants in crops?</q>\n",
      "3. <q>What are the benefits of consuming organic foods with higher antioxidant concentrations?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the higher antioxidant concentrations and lower incidence of pesticide residues in organically grown crops, and the potential benefits of consuming these foods.\n",
      "['What are the nutritional differences between organic and non-organic crops?', 'How do organic farming practices affect the concentration of antioxidants in crops?', 'What are the benefits of consuming organic foods with higher antioxidant concentrations?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the association between adherence to a Mediterranean diet and the risk of colorectal cancer?</q>\n",
      "2. <q>Can adherence to a Mediterranean diet reduce the risk of prostate cancer? If so, by how much?</q>\n",
      "3. <q>What is the effect of high adherence to a Mediterranean diet on the risk of overall cancer mortality?</q>\n",
      "\n",
      "These queries are answerable from Document 2, as the document discusses the association between adherence to a Mediterranean diet and the risk of various types of cancer, including colorectal, prostate, and overall cancer mortality.\n",
      "['What is the association between adherence to a Mediterranean diet and the risk of colorectal cancer?', 'Can adherence to a Mediterranean diet reduce the risk of prostate cancer? If so, by how much?', 'What is the effect of high adherence to a Mediterranean diet on the risk of overall cancer mortality?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the prevalence of monoclonal gammopathy of undetermined significance (MGUS) in healthy adults?</q>\n",
      "2. <q>Can monoclonal gammopathy of undetermined significance (MGUS) be detected before the onset of multiple myeloma?</q>\n",
      "3. <q>What are the patterns of monoclonal immunoglobulin abnormalities prior to multiple myeloma diagnosis?</q>\n",
      "\n",
      "These queries are based on the information presented in Document 2, which discusses the prevalence and characteristics of MGUS in a prospective study of healthy adults, and the potential for MGUS to precede the development of multiple myeloma.\n",
      "['What is the prevalence of monoclonal gammopathy of undetermined significance (MGUS) in healthy adults?', 'Can monoclonal gammopathy of undetermined significance (MGUS) be detected before the onset of multiple myeloma?', 'What are the patterns of monoclonal immunoglobulin abnormalities prior to multiple myeloma diagnosis?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the current recommendations for HPV vaccination for girls and boys?</q>\n",
      "2. <q>Are there any new patient-applied treatment options for genital warts?</q>\n",
      "3. <q>What are the updated counseling guidelines for patients on HPV, genital warts, and cervical cancer screening?</q>\n",
      "\n",
      "These queries are answerable from Document 2, as the document provides information on the current recommendations for HPV vaccination, new patient-applied treatment options for genital warts, and updated counseling guidelines for patients on HPV, genital warts, and cervical cancer screening.\n",
      "['What are the current recommendations for HPV vaccination for girls and boys?', 'Are there any new patient-applied treatment options for genital warts?', 'What are the updated counseling guidelines for patients on HPV, genital warts, and cervical cancer screening?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the best sources of vitamin B12 for vegans?</q>\n",
      "2. <q>How does the bioavailability of vitamin B12 vary in different types of meat?</q>\n",
      "3. <q>Can I get enough vitamin B12 from fortified breakfast cereals?</q>\n",
      "\n",
      "These queries are based on the information in Document 2 about the sources and bioavailability of vitamin B12, and are likely to be answerable from the document.\n",
      "['What are the best sources of vitamin B12 for vegans?', 'How does the bioavailability of vitamin B12 vary in different types of meat?', 'Can I get enough vitamin B12 from fortified breakfast cereals?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the relationship between diet and lifestyle and the risk of type 2 diabetes in women?</q>\n",
      "2. <q>How can I reduce my risk of developing type 2 diabetes through diet and lifestyle changes?</q>\n",
      "3. <q>What are the specific dietary and lifestyle factors that contribute to the development of type 2 diabetes in women?</q>\n",
      "\n",
      "These queries are based on the information provided in Document 2, which suggests that diet, lifestyle, and body mass index are important risk factors for the development of type 2 diabetes in women. The queries also reflect the findings of the study, which suggest that a healthy lifestyle and diet can significantly reduce the risk of developing type 2 diabetes.\n",
      "['What is the relationship between diet and lifestyle and the risk of type 2 diabetes in women?', 'How can I reduce my risk of developing type 2 diabetes through diet and lifestyle changes?', 'What are the specific dietary and lifestyle factors that contribute to the development of type 2 diabetes in women?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the average mercury levels in women of child-bearing age in the Florida Panhandle?</q>\n",
      "2. <q>How many women in the Florida Panhandle are unaware of fish consumption advisories due to mercury contamination?</q>\n",
      "3. <q>What are the fish consumption practices of women in the Florida Panhandle, and how do they relate to mercury exposure?</q>\n",
      "\n",
      "These queries are answerable from Document 2, as the document discusses the mercury levels in women of child-bearing age in the Florida Panhandle, the awareness of fish consumption advisories among these women, and their fish consumption practices.\n",
      "['What are the average mercury levels in women of child-bearing age in the Florida Panhandle?', 'How many women in the Florida Panhandle are unaware of fish consumption advisories due to mercury contamination?', 'What are the fish consumption practices of women in the Florida Panhandle, and how do they relate to mercury exposure?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the potential effects of dietary microparticles on the immune system of the gastrointestinal tract?</q>\n",
      "2. <q>How do exogenous dietary microparticles differ from endogenous microparticles in terms of their impact on the gastrointestinal tract?</q>\n",
      "3. <q>What is the role of M-cell rich lymphoid aggregates in the absorption of dietary microparticles in the gastrointestinal tract?</q>\n",
      "\n",
      "These queries are based on the information provided in Document 2, specifically regarding the impact of dietary microparticles on the gastrointestinal tract, the difference between endogenous and exogenous microparticles, and the role of M-cell rich lymphoid aggregates in the absorption of microparticles.\n",
      "['What are the potential effects of dietary microparticles on the immune system of the gastrointestinal tract?', 'How do exogenous dietary microparticles differ from endogenous microparticles in terms of their impact on the gastrointestinal tract?', 'What is the role of M-cell rich lymphoid aggregates in the absorption of dietary microparticles in the gastrointestinal tract?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the recommended treatments for nausea and migraines during pregnancy, according to health food store employees in the Phoenix area?</q>\n",
      "2. <q>How many health food store employees in the Phoenix area recommend the use of ginger for nausea and vomiting during pregnancy?</q>\n",
      "3. <q>Are there any dietary supplements that are contraindicated for use during pregnancy, according to health food store employees in the Phoenix area?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, but do not directly copy any text from the document.\n",
      "['What are the recommended treatments for nausea and migraines during pregnancy, according to health food store employees in the Phoenix area?', 'How many health food store employees in the Phoenix area recommend the use of ginger for nausea and vomiting during pregnancy?', 'Are there any dietary supplements that are contraindicated for use during pregnancy, according to health food store employees in the Phoenix area?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How does consuming golden kiwifruits affect DNA damage and repair?</q>\n",
      "2. <q>What is the effect of golden kiwifruit supplementation on plasma triglycerides and platelet aggregation?</q>\n",
      "3. <q>Can consuming golden kiwifruits improve antioxidant status and resistance towards oxidative damage?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, and do not use any text from the example provided.\n",
      "['How does consuming golden kiwifruits affect DNA damage and repair?', 'What is the effect of golden kiwifruit supplementation on plasma triglycerides and platelet aggregation?', 'Can consuming golden kiwifruits improve antioxidant status and resistance towards oxidative damage?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the best meditation practice for family dementia caregivers with depressive symptoms?</q>\n",
      "2. <q>How does brief daily yogic meditation affect cognitive functioning in family dementia caregivers with mild depressive symptoms?</q>\n",
      "3. <q>Can telomerase activity be increased through meditation practices in family dementia caregivers with depressive symptoms?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and are not based on the text of the example provided.\n",
      "['What is the best meditation practice for family dementia caregivers with depressive symptoms?', 'How does brief daily yogic meditation affect cognitive functioning in family dementia caregivers with mild depressive symptoms?', 'Can telomerase activity be increased through meditation practices in family dementia caregivers with depressive symptoms?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How does yeast beta-glucan help maintain the body's defense against pathogens?</q>\n",
      "2. <q>What is the effectiveness of yeast beta-glucan in reducing common cold episodes in healthy individuals?</q>\n",
      "3. <q>How does yeast beta-glucan prepare the body to defend against invading pathogens?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "[\"How does yeast beta-glucan help maintain the body's defense against pathogens?\", 'What is the effectiveness of yeast beta-glucan in reducing common cold episodes in healthy individuals?', 'How does yeast beta-glucan prepare the body to defend against invading pathogens?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated from Document 2:\n",
      "\n",
      "1. <q>What percentage of meat products in Tucson, Arizona contained toxigenic Clostridium difficile strains in 2007?</q>\n",
      "2. <q>What are the two most common ribotypes of Clostridium difficile found in retail meat products in the USA in 2007?</q>\n",
      "3. <q>How might food products contribute to the transmission of Clostridium difficile between species?</q>\n",
      "['What percentage of meat products in Tucson, Arizona contained toxigenic Clostridium difficile strains in 2007?', 'What are the two most common ribotypes of Clostridium difficile found in retail meat products in the USA in 2007?', 'How might food products contribute to the transmission of Clostridium difficile between species?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the optimal ratio of beans to white rice for lowering cardiometabolic risk factors?</q>\n",
      "2. <q>How does substituting beans for white rice affect metabolic syndrome risk in adults?</q>\n",
      "3. <q>What are the health benefits of increasing the intake of beans in a staple dietary pattern like Costa Rican adults follow?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the association between intake of white rice and beans and cardiometabolic risk factors in Costa Rican adults. The queries are designed to elicit information about the optimal ratio of beans to white rice for lowering risk factors, the effects of substituting beans for white rice on metabolic syndrome risk, and the health benefits of increasing bean intake in a staple dietary pattern.\n",
      "['What is the optimal ratio of beans to white rice for lowering cardiometabolic risk factors?', 'How does substituting beans for white rice affect metabolic syndrome risk in adults?', 'What are the health benefits of increasing the intake of beans in a staple dietary pattern like Costa Rican adults follow?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Based on the content of Document 2, the following are three search queries that can be generated:\n",
      "\n",
      "1. <q>What are the molecular mechanisms of black raspberry chemoprevention against oral cancer?</q>\n",
      "2. <q>How do ferulic acid and beta-sitosterol inhibit the growth of premalignant and malignant human oral cell lines?</q>\n",
      "3. <q>What are the effects of the RO-ET fraction on the cell cycle distribution of premalignant and malignant human oral cell lines?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from Example.\n",
      "['What are the molecular mechanisms of black raspberry chemoprevention against oral cancer?', 'How do ferulic acid and beta-sitosterol inhibit the growth of premalignant and malignant human oral cell lines?', 'What are the effects of the RO-ET fraction on the cell cycle distribution of premalignant and malignant human oral cell lines?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the recommended daily intakes of phytonutrients for adults in the United States?</q>\n",
      "2. <q>How do food sources contribute to total phytonutrient intakes in adults who consume a diet rich in fruits and vegetables?</q>\n",
      "3. <q>What are the top five food sources for five common phytonutrients (α-carotene, β-cryptoxanthin, lycopene, hesperetin, and ellagic acid) in the diets of adults who meet dietary recommendations for fruit and vegetable consumption?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What are the recommended daily intakes of phytonutrients for adults in the United States?', 'How do food sources contribute to total phytonutrient intakes in adults who consume a diet rich in fruits and vegetables?', 'What are the top five food sources for five common phytonutrients (α-carotene, β-cryptoxanthin, lycopene, hesperetin, and ellagic acid) in the diets of adults who meet dietary recommendations for fruit and vegetable consumption?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the effect of cinnamon supplementation on blood pressure in type 2 diabetic patients?</q>\n",
      "2. <q>Can cinnamon lower HbA1c levels in people with poorly controlled type 2 diabetes?</q>\n",
      "3. <q>Are there any dietary supplements that can help regulate blood glucose and blood pressure levels in type 2 diabetes patients?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What is the effect of cinnamon supplementation on blood pressure in type 2 diabetic patients?', 'Can cinnamon lower HbA1c levels in people with poorly controlled type 2 diabetes?', 'Are there any dietary supplements that can help regulate blood glucose and blood pressure levels in type 2 diabetes patients?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the safety concerns for women when taking statins?</q>\n",
      "2. <q>Why is there a lack of evidence-based research on statin use for women?</q>\n",
      "3. <q>How can women make informed decisions about statin use, considering their unique health needs and concerns?</q>\n",
      "['What are the safety concerns for women when taking statins?', 'Why is there a lack of evidence-based research on statin use for women?', 'How can women make informed decisions about statin use, considering their unique health needs and concerns?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the association between caffeine consumption and the risk of primary open-angle glaucoma?</q>\n",
      "2. <q>Does family history of glaucoma affect the relationship between caffeine intake and risk of primary open-angle glaucoma?</q>\n",
      "3. <q>What is the optimal amount of caffeine intake to reduce the risk of primary open-angle glaucoma?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What is the association between caffeine consumption and the risk of primary open-angle glaucoma?', 'Does family history of glaucoma affect the relationship between caffeine intake and risk of primary open-angle glaucoma?', 'What is the optimal amount of caffeine intake to reduce the risk of primary open-angle glaucoma?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Based on the content of Document 2, here are three search queries that could be used to retrieve relevant information from the document:\n",
      "\n",
      "1. <q>What are the potential risks of heavy metal toxicity from using Ayurvedic herbal medicine products?</q>\n",
      "2. <q>How common is the presence of heavy metals in Ayurvedic herbal medicine products manufactured in South Asia?</q>\n",
      "3. <q>What are the regulatory standards for heavy metal content in Ayurvedic herbal medicine products, and how do they compare to the levels found in products available in Boston-area stores?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What are the potential risks of heavy metal toxicity from using Ayurvedic herbal medicine products?', 'How common is the presence of heavy metals in Ayurvedic herbal medicine products manufactured in South Asia?', 'What are the regulatory standards for heavy metal content in Ayurvedic herbal medicine products, and how do they compare to the levels found in products available in Boston-area stores?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Based on the content of Document 2, here are three search queries that could be used to retrieve relevant information from other sources:\n",
      "\n",
      "1. <q>Effects of ractopamine on aggression in finishing pigs</q>\n",
      "2. <q>Ractopamine-induced changes in brain amine concentrations in pigs</q>\n",
      "3. <q>Social rank and aggression in finishing pigs fed ractopamine</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['Effects of ractopamine on aggression in finishing pigs', 'Ractopamine-induced changes in brain amine concentrations in pigs', 'Social rank and aggression in finishing pigs fed ractopamine']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the symptoms of cow's milk intolerance in children?</q>\n",
      "2. <q>Can chronic constipation in children be caused by intolerance to cow's milk?</q>\n",
      "3. <q>What is the best treatment for chronic constipation in children with suspected cow's milk intolerance?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and are not based on the text of the example provided.\n",
      "[\"What are the symptoms of cow's milk intolerance in children?\", \"Can chronic constipation in children be caused by intolerance to cow's milk?\", \"What is the best treatment for chronic constipation in children with suspected cow's milk intolerance?\"]\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the mechanism of skeletal muscle damage induced by statin therapy?</q>\n",
      "2. <q>Can statin therapy cause muscle damage without myalgia?</q>\n",
      "3. <q>What is the characteristic pattern of skeletal muscle damage induced by statin therapy?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the ultrastructural damage caused by statin therapy in skeletal muscle, even in patients without myalgia. The queries aim to elicit information about the mechanism of this damage and the specific characteristics of the damage that can be used to identify it.\n",
      "['What is the mechanism of skeletal muscle damage induced by statin therapy?', 'Can statin therapy cause muscle damage without myalgia?', 'What is the characteristic pattern of skeletal muscle damage induced by statin therapy?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the acute effects of caffeinated and decaffeinated espresso coffee on endothelial function in healthy subjects?</q>\n",
      "2. <q>How does the anti-oxidant capacity of caffeinated and decaffeinated coffee affect endothelial function?</q>\n",
      "3. <q>What are the chronic effects of coffee consumption on health, and how do they compare to the acute effects observed in the study?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, and do not use any text from Example.\n",
      "['What are the acute effects of caffeinated and decaffeinated espresso coffee on endothelial function in healthy subjects?', 'How does the anti-oxidant capacity of caffeinated and decaffeinated coffee affect endothelial function?', 'What are the chronic effects of coffee consumption on health, and how do they compare to the acute effects observed in the study?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How can physicians enhance their credibility and motivate patients to adopt healthy habits?</q>\n",
      "2. <q>What are the benefits of physicians disclosing their personal healthy behaviors to their patients?</q>\n",
      "3. <q>How can educational institutions help health professionals-in-training to practice and demonstrate healthy personal lifestyles?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which suggests that physicians' disclosure of their own healthy behaviors can enhance their credibility and motivate patients to adopt healthier habits. The queries are also relevant to the topic of health education and the role of physicians in promoting healthy lifestyles.\n",
      "['How can physicians enhance their credibility and motivate patients to adopt healthy habits?', 'What are the benefits of physicians disclosing their personal healthy behaviors to their patients?', 'How can educational institutions help health professionals-in-training to practice and demonstrate healthy personal lifestyles?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How does flaxseed lignan extract affect plasma cholesterol levels?</q>\n",
      "2. <q>What is the optimal dosage of dietary flaxseed lignan extract for lowering plasma glucose levels?</q>\n",
      "3. <q>Can flaxseed lignan extract affect plasma concentrations of secoisolariciresinol and enterodiol?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from Example.\n",
      "['How does flaxseed lignan extract affect plasma cholesterol levels?', 'What is the optimal dosage of dietary flaxseed lignan extract for lowering plasma glucose levels?', 'Can flaxseed lignan extract affect plasma concentrations of secoisolariciresinol and enterodiol?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the potential health benefits of Lycium barbarum and L. chinense for anti-aging and well-being?</q>\n",
      "2. <q>Can the use of Goji products interact with certain medications, such as warfarin?</q>\n",
      "3. <q>What are the traditional uses of Lycium barbarum and L. chinense in East Asian folk medicine, and how have these uses evolved with the recent popularity of Goji products?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, but do not directly copy any text from the document.\n",
      "['What are the potential health benefits of Lycium barbarum and L. chinense for anti-aging and well-being?', 'Can the use of Goji products interact with certain medications, such as warfarin?', 'What are the traditional uses of Lycium barbarum and L. chinense in East Asian folk medicine, and how have these uses evolved with the recent popularity of Goji products?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated from Document 2:\n",
      "\n",
      "1. <q>What is the optimal dietary fibre intake for maintaining rapid bowel transit times?</q>\n",
      "2. <q>How does a fibre-depleted diet affect the colon and bowel transit times?</q>\n",
      "3. <q>Can diverticular disease affect bowel transit times, and if so, how?</q>\n",
      "\n",
      "These queries are based on the information presented in Document 2, specifically the relationship between dietary fibre intake and bowel transit times, the differences in bowel transit times between vegetarians and non-vegetarians, and the potential effects of a fibre-depleted diet on the colon and bowel transit times.\n",
      "['What is the optimal dietary fibre intake for maintaining rapid bowel transit times?', 'How does a fibre-depleted diet affect the colon and bowel transit times?', 'Can diverticular disease affect bowel transit times, and if so, how?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the causes of vitamin A deficiency in South Asian developing countries?</q>\n",
      "2. <q>How can we improve vitamin A status in South Asian countries with poor governmental support?</q>\n",
      "3. <q>What are the long-term effects of vitamin A deficiency on childhood blindness and morbidity in South Asia?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and are not based on the text of Example.\n",
      "['What are the causes of vitamin A deficiency in South Asian developing countries?', 'How can we improve vitamin A status in South Asian countries with poor governmental support?', 'What are the long-term effects of vitamin A deficiency on childhood blindness and morbidity in South Asia?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the long-term effects of comprehensive lifestyle modification on hypertension control?</q>\n",
      "2. <q>How do multicomponent behavioral interventions compare to advice only in improving lifestyle changes and blood pressure control?</q>\n",
      "3. <q>What are the optimal dietary and physical activity modifications for reducing blood pressure in individuals with prehypertension and stage 1 hypertension?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What are the long-term effects of comprehensive lifestyle modification on hypertension control?', 'How do multicomponent behavioral interventions compare to advice only in improving lifestyle changes and blood pressure control?', 'What are the optimal dietary and physical activity modifications for reducing blood pressure in individuals with prehypertension and stage 1 hypertension?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the effect of Hibiscus sabdariffa L. aqueous extract on systemic antioxidant potential in healthy subjects?</q>\n",
      "2. <q>How does the consumption of Hibiscus sabdariffa L. aqueous extract affect the excretion of antioxidants and oxidative stress biomarkers in urine?</q>\n",
      "3. <q>What are the biokinetic variables that are affected by the consumption of Hibiscus sabdariffa L. aqueous extract in healthy subjects?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What is the effect of Hibiscus sabdariffa L. aqueous extract on systemic antioxidant potential in healthy subjects?', 'How does the consumption of Hibiscus sabdariffa L. aqueous extract affect the excretion of antioxidants and oxidative stress biomarkers in urine?', 'What are the biokinetic variables that are affected by the consumption of Hibiscus sabdariffa L. aqueous extract in healthy subjects?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the association between nut consumption and pancreatic cancer risk in women?</q>\n",
      "2. <q>How does frequent nut consumption affect the risk of pancreatic cancer in women, independent of other risk factors?</q>\n",
      "3. <q>Can a serving size of 28g (1oz) of nuts per week reduce the risk of pancreatic cancer in women?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, and do not use any text from Example.\n",
      "['What is the association between nut consumption and pancreatic cancer risk in women?', 'How does frequent nut consumption affect the risk of pancreatic cancer in women, independent of other risk factors?', 'Can a serving size of 28g (1oz) of nuts per week reduce the risk of pancreatic cancer in women?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Based on the content of Document 2, the following are three search queries that could be used to retrieve relevant information from the document:\n",
      "\n",
      "1. <q>What is the incidence of probable dementia and mild cognitive impairment in postmenopausal women treated with conjugated equine estrogens?</q>\n",
      "2. <q>How does the risk of dementia and mild cognitive impairment differ between women treated with conjugated equine estrogens alone and those treated with both estrogen and progestin?</q>\n",
      "3. <q>What are the factors that contribute to the development of probable dementia and mild cognitive impairment in postmenopausal women, and how do they interact with hormone therapy?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['What is the incidence of probable dementia and mild cognitive impairment in postmenopausal women treated with conjugated equine estrogens?', 'How does the risk of dementia and mild cognitive impairment differ between women treated with conjugated equine estrogens alone and those treated with both estrogen and progestin?', 'What are the factors that contribute to the development of probable dementia and mild cognitive impairment in postmenopausal women, and how do they interact with hormone therapy?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the typical mercury concentrations found in the hair of diseased people in Japan?</q>\n",
      "2. <q>How do the uptake routes and metabolic mechanisms of mercury differ in diseased people compared to healthy individuals?</q>\n",
      "3. <q>Can high mercury concentrations in hair be a factor in the development of certain diseases, such as atopic dermatitis or osteoporosis?</q>\n",
      "['What are the typical mercury concentrations found in the hair of diseased people in Japan?', 'How do the uptake routes and metabolic mechanisms of mercury differ in diseased people compared to healthy individuals?', 'Can high mercury concentrations in hair be a factor in the development of certain diseases, such as atopic dermatitis or osteoporosis?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the potential therapeutic applications of bioactive plant lignans?</q>\n",
      "2. <q>Can lignans be used in cancer chemotherapy? What is the current research on this topic?</q>\n",
      "3. <q>What are the structural diversity and molecular backbone of lignans, and how do they contribute to their pharmacological effects?</q>\n",
      "\n",
      "These queries are answerable from Document 2, as the document discusses the various pharmacological effects of lignans, including their potential use in cancer chemotherapy, and provides information on the structural diversity and molecular backbone of lignans.\n",
      "['What are the potential therapeutic applications of bioactive plant lignans?', 'Can lignans be used in cancer chemotherapy? What is the current research on this topic?', 'What are the structural diversity and molecular backbone of lignans, and how do they contribute to their pharmacological effects?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the association between vegetable consumption and depression risk in older adults?</q>\n",
      "2. <q>Can frequent consumption of eggs reduce the risk of depression in older adults?</q>\n",
      "3. <q>How does cognitive status affect the relationship between food consumption and depression risk in older adults?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not rely on the text from Example.\n",
      "['What is the association between vegetable consumption and depression risk in older adults?', 'Can frequent consumption of eggs reduce the risk of depression in older adults?', 'How does cognitive status affect the relationship between food consumption and depression risk in older adults?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>Can eating nuts regularly help with weight loss or weight management?</q>\n",
      "2. <q>Are there any specific types of nuts that are more effective at promoting weight loss?</q>\n",
      "3. <q>How many calories in nuts are considered \"safe\" for weight loss, and what is the optimal serving size for weight management?</q>\n",
      "\n",
      "These queries are based on the information in Document 2 regarding the health benefits of nut consumption and the misconception that nut consumption can lead to weight gain. The queries are designed to seek information on the relationship between nut consumption and body weight gain, as well as the effectiveness of different types of nuts for weight loss and weight management.\n",
      "['Can eating nuts regularly help with weight loss or weight management?', 'Are there any specific types of nuts that are more effective at promoting weight loss?', 'How many calories in nuts are considered \"safe\" for weight loss, and what is the optimal serving size for weight management?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the molecular mechanisms by which curcumin inhibits carcinogenesis in various cancers?</q>\n",
      "2. <q>How does curcumin modulate cell signaling pathways to induce protective effects against cancers?</q>\n",
      "3. <q>What is the current understanding of the therapeutic value of curcumin in the treatment and prevention of a wide range of cancers?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2, and do not use any text from the example provided.\n",
      "['What are the molecular mechanisms by which curcumin inhibits carcinogenesis in various cancers?', 'How does curcumin modulate cell signaling pathways to induce protective effects against cancers?', 'What is the current understanding of the therapeutic value of curcumin in the treatment and prevention of a wide range of cancers?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How does dietary cholesterol affect LDL cholesterol and HDL cholesterol levels in elderly individuals?</q>\n",
      "2. <q>Can consuming eggs increase the risk of heart disease in an elderly population?</q>\n",
      "3. <q>What are the effects of a cholesterol-free, fat-free egg substitute on plasma lipids and LDL atherogenicity in elderly individuals?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and are not based on the text of Example.\n",
      "['How does dietary cholesterol affect LDL cholesterol and HDL cholesterol levels in elderly individuals?', 'Can consuming eggs increase the risk of heart disease in an elderly population?', 'What are the effects of a cholesterol-free, fat-free egg substitute on plasma lipids and LDL atherogenicity in elderly individuals?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the role of free radicals in regulating gene expression in skin and muscle tissues?</q>\n",
      "2. <q>How do antioxidant nutrients influence the adaptation of cells to oxidative stress in skin and muscle?</q>\n",
      "3. <q>Can free radicals act as signals for cellular adaptation in skin and muscle tissues, and if so, how?</q>\n",
      "['What is the role of free radicals in regulating gene expression in skin and muscle tissues?', 'How do antioxidant nutrients influence the adaptation of cells to oxidative stress in skin and muscle?', 'Can free radicals act as signals for cellular adaptation in skin and muscle tissues, and if so, how?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the effect of a lactovegetarian diet on faecal elimination of lead and cadmium?</q>\n",
      "2. <q>How does the water content of faeces affect the elimination of lead and cadmium?</q>\n",
      "3. <q>What is the range of faecal elimination of lead and cadmium in subjects on a mixed diet versus a lactovegetarian diet?</q>\n",
      "['What is the effect of a lactovegetarian diet on faecal elimination of lead and cadmium?', 'How does the water content of faeces affect the elimination of lead and cadmium?', 'What is the range of faecal elimination of lead and cadmium in subjects on a mixed diet versus a lactovegetarian diet?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on the content of Document 2:\n",
      "\n",
      "1. <q>What is the prevalence of ideal cardiovascular health behaviors among young adults in the US?</q>\n",
      "2. <q>How do the prevalence of ideal cardiovascular health factors vary by age and sex in the US adult population?</q>\n",
      "3. <q>What percentage of US adults exhibit ideal cardiovascular health for all seven metrics, according to the National Health and Nutrition Examination Surveys (NHANES) 2003-2008?</q>\n",
      "\n",
      "Note that these queries are based on the information provided in Document 2, and do not use any text from the example provided.\n",
      "['What is the prevalence of ideal cardiovascular health behaviors among young adults in the US?', 'How do the prevalence of ideal cardiovascular health factors vary by age and sex in the US adult population?', 'What percentage of US adults exhibit ideal cardiovascular health for all seven metrics, according to the National Health and Nutrition Examination Surveys (NHANES) 2003-2008?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the possible causes of the advance in timing of puberty in girls in the USA?</q>\n",
      "2. <q>How does early maturing affect the health and well-being of girls and boys in adolescence and adulthood?</q>\n",
      "3. <q>What is the relationship between hyperinsulinaemia and the onset of puberty, and how might this factor into the advancement of puberty in the USA?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the controversy surrounding the advance in the timing of puberty in girls in the USA, and explores possible causative factors such as hyperinsulinaemia and insulin resistance. The queries do not rely on text from Example, and are instead based on the unique content of Document 2.\n",
      "['What are the possible causes of the advance in timing of puberty in girls in the USA?', 'How does early maturing affect the health and well-being of girls and boys in adolescence and adulthood?', 'What is the relationship between hyperinsulinaemia and the onset of puberty, and how might this factor into the advancement of puberty in the USA?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the most common extraintestinal infections caused by Escherichia coli?</q>\n",
      "2. <q>How does the incidence of extraintestinal infections due to E. coli vary by geographic region?</q>\n",
      "3. <q>What are the economic and medical impacts of antimicrobial resistance in E. coli infections?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which highlights the underappreciation of E. coli as an extraintestinal pathogen, the high incidence and associated morbidity and mortality of extraintestinal infections, and the need for new treatments and prevention measures to address the disease burden.\n",
      "['What are the most common extraintestinal infections caused by Escherichia coli?', 'How does the incidence of extraintestinal infections due to E. coli vary by geographic region?', 'What are the economic and medical impacts of antimicrobial resistance in E. coli infections?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the effect of cigarette smoking on semen quality in men with idiopathic infertility?</q>\n",
      "2. <q>How does the amount of cigarette smoking affect sperm concentration and fertility index in men with idiopathic infertility?</q>\n",
      "3. <q>Can cigarette smoking cause sperm apoptosis, necrosis, or immaturity in men with idiopathic infertility?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from Example.\n",
      "['What is the effect of cigarette smoking on semen quality in men with idiopathic infertility?', 'How does the amount of cigarette smoking affect sperm concentration and fertility index in men with idiopathic infertility?', 'Can cigarette smoking cause sperm apoptosis, necrosis, or immaturity in men with idiopathic infertility?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the potential health benefits of dark chocolate for cardiovascular disease risk reduction?</q>\n",
      "2. <q>How does the production process of cocoa affect its potential health benefits?</q>\n",
      "3. <q>Can cocoa consumption improve endothelial function and reduce the risk of cardiovascular disease?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, which discusses the potential health benefits of cocoa and dark chocolate, the possible mechanisms by which they may confer these benefits, and the considerations for using cocoa as a dietary supplement.\n",
      "['What are the potential health benefits of dark chocolate for cardiovascular disease risk reduction?', 'How does the production process of cocoa affect its potential health benefits?', 'Can cocoa consumption improve endothelial function and reduce the risk of cardiovascular disease?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated from Document 2:\n",
      "\n",
      "1. <q>What was the therapeutic index of dinitrophenol in the early 1930s?</q>\n",
      "2. <q>Why did mainstream physicians stop using dinitrophenol as a weight-loss drug?</q>\n",
      "3. <q>What legislation was passed in 1938 to regulate the sale of dinitrophenol in the United States?</q>\n",
      "['What was the therapeutic index of dinitrophenol in the early 1930s?', 'Why did mainstream physicians stop using dinitrophenol as a weight-loss drug?', 'What legislation was passed in 1938 to regulate the sale of dinitrophenol in the United States?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the difference between ductal carcinoma in situ (DCIS) and invasive breast cancer?</q>\n",
      "2. <q>How is ductal carcinoma in situ (DCIS) classified, and what are the different types?</q>\n",
      "3. <q>What is the natural history of ductal carcinoma in situ (DCIS), and how does it progress to invasive breast cancer?</q>\n",
      "\n",
      "These queries are answerable from Document 2, as the document provides information on the terminology, classification, and natural history of DCIS.\n",
      "['What is the difference between ductal carcinoma in situ (DCIS) and invasive breast cancer?', 'How is ductal carcinoma in situ (DCIS) classified, and what are the different types?', 'What is the natural history of ductal carcinoma in situ (DCIS), and how does it progress to invasive breast cancer?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, I can help you generate three search queries based on Document 2. Here are the queries:\n",
      "\n",
      "1. <q>What is the difference in visceral sensitivity between patients with functional abdominal pain syndrome (FAPS) and healthy controls?</q>\n",
      "2. <q>How does rectal balloon distention affect patients with irritable bowel syndrome (IBS) and FAPS?</q>\n",
      "3. <q>What is the significance of altered rectal compliance in patients with FAPS?</q>\n",
      "\n",
      "These queries are based on the content of Document 2, specifically focusing on the differences in visceral sensitivity between FAPS and healthy controls, the effects of rectal balloon distention on IBS and FAPS patients, and the significance of altered rectal compliance in FAPS patients.\n",
      "['What is the difference in visceral sensitivity between patients with functional abdominal pain syndrome (FAPS) and healthy controls?', 'How does rectal balloon distention affect patients with irritable bowel syndrome (IBS) and FAPS?', 'What is the significance of altered rectal compliance in patients with FAPS?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the prevalence of celiac disease in the United States?</q>\n",
      "2. <q>How many people in the United States are unaware they have celiac disease?</q>\n",
      "3. <q>What is the prevalence of celiac disease among non-Hispanic whites in the United States?</q>\n",
      "\n",
      "Note that these queries are based on the information provided in Document 2, and do not use any text from the example provided.\n",
      "['What is the prevalence of celiac disease in the United States?', 'How many people in the United States are unaware they have celiac disease?', 'What is the prevalence of celiac disease among non-Hispanic whites in the United States?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What is the historical evidence for the association between the Mediterranean diet and cardiovascular disease prevention?</q>\n",
      "2. <q>Can the Mediterranean diet reduce the risk of cardiovascular complications in non-Mediterranean populations?</q>\n",
      "3. <q>What are the long-term benefits of adhering to the Mediterranean diet for cardiovascular health, beyond just reducing the risk of cardiovascular disease?</q>\n",
      "['What is the historical evidence for the association between the Mediterranean diet and cardiovascular disease prevention?', 'Can the Mediterranean diet reduce the risk of cardiovascular complications in non-Mediterranean populations?', 'What are the long-term benefits of adhering to the Mediterranean diet for cardiovascular health, beyond just reducing the risk of cardiovascular disease?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the recent advances in natural and organic curing processes for processed meats in the United States?</q>\n",
      "2. <q>How have consumer demand and regulatory requirements influenced the development of natural and organic curing processes in the meat industry?</q>\n",
      "3. <q>What are some of the alternative ingredients and processes used in natural and organic curing processes to ensure microbiological safety and extend shelf life?</q>\n",
      "['What are the recent advances in natural and organic curing processes for processed meats in the United States?', 'How have consumer demand and regulatory requirements influenced the development of natural and organic curing processes in the meat industry?', 'What are some of the alternative ingredients and processes used in natural and organic curing processes to ensure microbiological safety and extend shelf life?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>What are the anti-inflammatory activities of curcumin-free turmeric?</q>\n",
      "2. <q>Can turmeric oil enhance the bioavailability of curcumin?</q>\n",
      "3. <q>What are the anticancer activities of individual components of turmeric, such as elemene and germacrone?</q>\n",
      "\n",
      "These queries are based on the information provided in Document 2, which discusses the anticancer and anti-inflammatory activities of curcumin-free turmeric and individual components of turmeric.\n",
      "['What are the anti-inflammatory activities of curcumin-free turmeric?', 'Can turmeric oil enhance the bioavailability of curcumin?', 'What are the anticancer activities of individual components of turmeric, such as elemene and germacrone?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>Can adenovirus infection cause obesity in humans?</q>\n",
      "2. <q>What is the relationship between SMAM-1 virus and human obesity?</q>\n",
      "3. <q>Can serum antibodies against SMAM-1 virus be used to diagnose obesity in humans?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['Can adenovirus infection cause obesity in humans?', 'What is the relationship between SMAM-1 virus and human obesity?', 'Can serum antibodies against SMAM-1 virus be used to diagnose obesity in humans?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Based on the content of Document 2, here are three search queries that could be used to retrieve relevant information from the document:\n",
      "\n",
      "1. <q>What is the relationship between meditation and telomerase activity?</q>\n",
      "2. <q>How does perceived control affect the relationship between meditation and stress?</q>\n",
      "3. <q>What is the role of mindfulness and purpose in life in mediating the effects of meditation on stress and well-being?</q>\n",
      "\n",
      "Note that these queries are based on the specific language and concepts used in Document 2, and may not be relevant to other documents or sources.\n",
      "['What is the relationship between meditation and telomerase activity?', 'How does perceived control affect the relationship between meditation and stress?', 'What is the role of mindfulness and purpose in life in mediating the effects of meditation on stress and well-being?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "  Sure, here are three search queries that can be generated based on Document 2:\n",
      "\n",
      "1. <q>How does consuming blueberries with a high-carbohydrate breakfast affect oxidation levels?</q>\n",
      "2. <q>What is the optimal amount of blueberries to consume with a low-fat breakfast for oxidative protection?</q>\n",
      "3. <q>Can consuming blueberries with a high-carbohydrate, low-fat breakfast improve serum antioxidant capacity?</q>\n",
      "\n",
      "Note that these queries are based on the content of Document 2 and do not use any text from the example provided.\n",
      "['How does consuming blueberries with a high-carbohydrate breakfast affect oxidation levels?', 'What is the optimal amount of blueberries to consume with a low-fat breakfast for oxidative protection?', 'Can consuming blueberries with a high-carbohydrate, low-fat breakfast improve serum antioxidant capacity?']\n",
      "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "extracted_questions = extract_questions_from_generations(kb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>_id</th>\n",
       "      <th>title</th>\n",
       "      <th>text</th>\n",
       "      <th>paragraph_id</th>\n",
       "      <th>chunk_text</th>\n",
       "      <th>chunk_id</th>\n",
       "      <th>title_text_concat</th>\n",
       "      <th>chunk_title_text_concat</th>\n",
       "      <th>prompt</th>\n",
       "      <th>generated_text</th>\n",
       "      <th>chunk_question</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MED-1710</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>Sugar intake in the United States has increase...</td>\n",
       "      <td>0</td>\n",
       "      <td>Sugar intake in the United States has increase...</td>\n",
       "      <td>0</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Sure, here are three search queries that can...</td>\n",
       "      <td>What are the health risks associated with cons...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MED-1710</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>Sugar intake in the United States has increase...</td>\n",
       "      <td>0</td>\n",
       "      <td>Sugar intake in the United States has increase...</td>\n",
       "      <td>0</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Sure, here are three search queries that can...</td>\n",
       "      <td>How does fructose intake from sugar-sweetened ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MED-1710</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>Sugar intake in the United States has increase...</td>\n",
       "      <td>0</td>\n",
       "      <td>Sugar intake in the United States has increase...</td>\n",
       "      <td>0</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Sure, here are three search queries that can...</td>\n",
       "      <td>What are the differences in metabolic effects ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MED-5032</td>\n",
       "      <td>Processed meats and risk of childhood leukemia...</td>\n",
       "      <td>The relation between the intake of certain foo...</td>\n",
       "      <td>1</td>\n",
       "      <td>The relation between the intake of certain foo...</td>\n",
       "      <td>0</td>\n",
       "      <td>Processed meats and risk of childhood leukemia...</td>\n",
       "      <td>Processed meats and risk of childhood leukemia...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Sure, here are three search queries that can...</td>\n",
       "      <td>What is the association between hot dog consum...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MED-5032</td>\n",
       "      <td>Processed meats and risk of childhood leukemia...</td>\n",
       "      <td>The relation between the intake of certain foo...</td>\n",
       "      <td>1</td>\n",
       "      <td>The relation between the intake of certain foo...</td>\n",
       "      <td>0</td>\n",
       "      <td>Processed meats and risk of childhood leukemia...</td>\n",
       "      <td>Processed meats and risk of childhood leukemia...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Sure, here are three search queries that can...</td>\n",
       "      <td>Do processed meats increase the risk of leukem...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>295</th>\n",
       "      <td>MED-1918</td>\n",
       "      <td>Intensive meditation training, immune cell tel...</td>\n",
       "      <td>BACKGROUND: Telomerase activity is a predictor...</td>\n",
       "      <td>98</td>\n",
       "      <td>BACKGROUND: Telomerase activity is a predictor...</td>\n",
       "      <td>0</td>\n",
       "      <td>Intensive meditation training, immune cell tel...</td>\n",
       "      <td>Intensive meditation training, immune cell tel...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Based on the content of Document 2, here are...</td>\n",
       "      <td>How does perceived control affect the relation...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>296</th>\n",
       "      <td>MED-1918</td>\n",
       "      <td>Intensive meditation training, immune cell tel...</td>\n",
       "      <td>BACKGROUND: Telomerase activity is a predictor...</td>\n",
       "      <td>98</td>\n",
       "      <td>BACKGROUND: Telomerase activity is a predictor...</td>\n",
       "      <td>0</td>\n",
       "      <td>Intensive meditation training, immune cell tel...</td>\n",
       "      <td>Intensive meditation training, immune cell tel...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Based on the content of Document 2, here are...</td>\n",
       "      <td>What is the role of mindfulness and purpose in...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>297</th>\n",
       "      <td>MED-1259</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>We sought to determine whether consumption of ...</td>\n",
       "      <td>99</td>\n",
       "      <td>We sought to determine whether consumption of ...</td>\n",
       "      <td>0</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Sure, here are three search queries that can...</td>\n",
       "      <td>How does consuming blueberries with a high-car...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298</th>\n",
       "      <td>MED-1259</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>We sought to determine whether consumption of ...</td>\n",
       "      <td>99</td>\n",
       "      <td>We sought to determine whether consumption of ...</td>\n",
       "      <td>0</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Sure, here are three search queries that can...</td>\n",
       "      <td>What is the optimal amount of blueberries to c...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>299</th>\n",
       "      <td>MED-1259</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>We sought to determine whether consumption of ...</td>\n",
       "      <td>99</td>\n",
       "      <td>We sought to determine whether consumption of ...</td>\n",
       "      <td>0</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>&lt;s&gt;[INST] &lt;&lt;SYS&gt;&gt;\\nYou are a data annotator tr...</td>\n",
       "      <td>Sure, here are three search queries that can...</td>\n",
       "      <td>Can consuming blueberries with a high-carbohyd...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>300 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          _id                                              title  \\\n",
       "0    MED-1710  Energy and Fructose From Beverages Sweetened W...   \n",
       "1    MED-1710  Energy and Fructose From Beverages Sweetened W...   \n",
       "2    MED-1710  Energy and Fructose From Beverages Sweetened W...   \n",
       "3    MED-5032  Processed meats and risk of childhood leukemia...   \n",
       "4    MED-5032  Processed meats and risk of childhood leukemia...   \n",
       "..        ...                                                ...   \n",
       "295  MED-1918  Intensive meditation training, immune cell tel...   \n",
       "296  MED-1918  Intensive meditation training, immune cell tel...   \n",
       "297  MED-1259  Consumption of blueberries with a high-carbohy...   \n",
       "298  MED-1259  Consumption of blueberries with a high-carbohy...   \n",
       "299  MED-1259  Consumption of blueberries with a high-carbohy...   \n",
       "\n",
       "                                                  text  paragraph_id  \\\n",
       "0    Sugar intake in the United States has increase...             0   \n",
       "1    Sugar intake in the United States has increase...             0   \n",
       "2    Sugar intake in the United States has increase...             0   \n",
       "3    The relation between the intake of certain foo...             1   \n",
       "4    The relation between the intake of certain foo...             1   \n",
       "..                                                 ...           ...   \n",
       "295  BACKGROUND: Telomerase activity is a predictor...            98   \n",
       "296  BACKGROUND: Telomerase activity is a predictor...            98   \n",
       "297  We sought to determine whether consumption of ...            99   \n",
       "298  We sought to determine whether consumption of ...            99   \n",
       "299  We sought to determine whether consumption of ...            99   \n",
       "\n",
       "                                            chunk_text  chunk_id  \\\n",
       "0    Sugar intake in the United States has increase...         0   \n",
       "1    Sugar intake in the United States has increase...         0   \n",
       "2    Sugar intake in the United States has increase...         0   \n",
       "3    The relation between the intake of certain foo...         0   \n",
       "4    The relation between the intake of certain foo...         0   \n",
       "..                                                 ...       ...   \n",
       "295  BACKGROUND: Telomerase activity is a predictor...         0   \n",
       "296  BACKGROUND: Telomerase activity is a predictor...         0   \n",
       "297  We sought to determine whether consumption of ...         0   \n",
       "298  We sought to determine whether consumption of ...         0   \n",
       "299  We sought to determine whether consumption of ...         0   \n",
       "\n",
       "                                     title_text_concat  \\\n",
       "0    Energy and Fructose From Beverages Sweetened W...   \n",
       "1    Energy and Fructose From Beverages Sweetened W...   \n",
       "2    Energy and Fructose From Beverages Sweetened W...   \n",
       "3    Processed meats and risk of childhood leukemia...   \n",
       "4    Processed meats and risk of childhood leukemia...   \n",
       "..                                                 ...   \n",
       "295  Intensive meditation training, immune cell tel...   \n",
       "296  Intensive meditation training, immune cell tel...   \n",
       "297  Consumption of blueberries with a high-carbohy...   \n",
       "298  Consumption of blueberries with a high-carbohy...   \n",
       "299  Consumption of blueberries with a high-carbohy...   \n",
       "\n",
       "                               chunk_title_text_concat  \\\n",
       "0    Energy and Fructose From Beverages Sweetened W...   \n",
       "1    Energy and Fructose From Beverages Sweetened W...   \n",
       "2    Energy and Fructose From Beverages Sweetened W...   \n",
       "3    Processed meats and risk of childhood leukemia...   \n",
       "4    Processed meats and risk of childhood leukemia...   \n",
       "..                                                 ...   \n",
       "295  Intensive meditation training, immune cell tel...   \n",
       "296  Intensive meditation training, immune cell tel...   \n",
       "297  Consumption of blueberries with a high-carbohy...   \n",
       "298  Consumption of blueberries with a high-carbohy...   \n",
       "299  Consumption of blueberries with a high-carbohy...   \n",
       "\n",
       "                                                prompt  \\\n",
       "0    <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "1    <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "2    <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "3    <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "4    <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "..                                                 ...   \n",
       "295  <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "296  <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "297  <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "298  <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "299  <s>[INST] <<SYS>>\\nYou are a data annotator tr...   \n",
       "\n",
       "                                        generated_text  \\\n",
       "0      Sure, here are three search queries that can...   \n",
       "1      Sure, here are three search queries that can...   \n",
       "2      Sure, here are three search queries that can...   \n",
       "3      Sure, here are three search queries that can...   \n",
       "4      Sure, here are three search queries that can...   \n",
       "..                                                 ...   \n",
       "295    Based on the content of Document 2, here are...   \n",
       "296    Based on the content of Document 2, here are...   \n",
       "297    Sure, here are three search queries that can...   \n",
       "298    Sure, here are three search queries that can...   \n",
       "299    Sure, here are three search queries that can...   \n",
       "\n",
       "                                        chunk_question  \n",
       "0    What are the health risks associated with cons...  \n",
       "1    How does fructose intake from sugar-sweetened ...  \n",
       "2    What are the differences in metabolic effects ...  \n",
       "3    What is the association between hot dog consum...  \n",
       "4    Do processed meats increase the risk of leukem...  \n",
       "..                                                 ...  \n",
       "295  How does perceived control affect the relation...  \n",
       "296  What is the role of mindfulness and purpose in...  \n",
       "297  How does consuming blueberries with a high-car...  \n",
       "298  What is the optimal amount of blueberries to c...  \n",
       "299  Can consuming blueberries with a high-carbohyd...  \n",
       "\n",
       "[300 rows x 11 columns]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kb = kb.merge(extracted_questions, how=\"left\", left_on=[\"paragraph_id\", \"chunk_id\"], right_on=[\"paragraph_id\", \"chunk_id\"])\n",
    "kb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>question</th>\n",
       "      <th>positive_chunk</th>\n",
       "      <th>positive_chunk_id</th>\n",
       "      <th>paragraph_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>What are the health risks associated with cons...</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>How does fructose intake from sugar-sweetened ...</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>What are the differences in metabolic effects ...</td>\n",
       "      <td>Energy and Fructose From Beverages Sweetened W...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>What is the association between hot dog consum...</td>\n",
       "      <td>Processed meats and risk of childhood leukemia...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Do processed meats increase the risk of leukem...</td>\n",
       "      <td>Processed meats and risk of childhood leukemia...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>295</th>\n",
       "      <td>How does perceived control affect the relation...</td>\n",
       "      <td>Intensive meditation training, immune cell tel...</td>\n",
       "      <td>0</td>\n",
       "      <td>98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>296</th>\n",
       "      <td>What is the role of mindfulness and purpose in...</td>\n",
       "      <td>Intensive meditation training, immune cell tel...</td>\n",
       "      <td>0</td>\n",
       "      <td>98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>297</th>\n",
       "      <td>How does consuming blueberries with a high-car...</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>0</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>298</th>\n",
       "      <td>What is the optimal amount of blueberries to c...</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>0</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>299</th>\n",
       "      <td>Can consuming blueberries with a high-carbohyd...</td>\n",
       "      <td>Consumption of blueberries with a high-carbohy...</td>\n",
       "      <td>0</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>300 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              question  \\\n",
       "0    What are the health risks associated with cons...   \n",
       "1    How does fructose intake from sugar-sweetened ...   \n",
       "2    What are the differences in metabolic effects ...   \n",
       "3    What is the association between hot dog consum...   \n",
       "4    Do processed meats increase the risk of leukem...   \n",
       "..                                                 ...   \n",
       "295  How does perceived control affect the relation...   \n",
       "296  What is the role of mindfulness and purpose in...   \n",
       "297  How does consuming blueberries with a high-car...   \n",
       "298  What is the optimal amount of blueberries to c...   \n",
       "299  Can consuming blueberries with a high-carbohyd...   \n",
       "\n",
       "                                        positive_chunk  positive_chunk_id  \\\n",
       "0    Energy and Fructose From Beverages Sweetened W...                  0   \n",
       "1    Energy and Fructose From Beverages Sweetened W...                  0   \n",
       "2    Energy and Fructose From Beverages Sweetened W...                  0   \n",
       "3    Processed meats and risk of childhood leukemia...                  0   \n",
       "4    Processed meats and risk of childhood leukemia...                  0   \n",
       "..                                                 ...                ...   \n",
       "295  Intensive meditation training, immune cell tel...                  0   \n",
       "296  Intensive meditation training, immune cell tel...                  0   \n",
       "297  Consumption of blueberries with a high-carbohy...                  0   \n",
       "298  Consumption of blueberries with a high-carbohy...                  0   \n",
       "299  Consumption of blueberries with a high-carbohy...                  0   \n",
       "\n",
       "     paragraph_id  \n",
       "0               0  \n",
       "1               0  \n",
       "2               0  \n",
       "3               1  \n",
       "4               1  \n",
       "..            ...  \n",
       "295            98  \n",
       "296            98  \n",
       "297            99  \n",
       "298            99  \n",
       "299            99  \n",
       "\n",
       "[300 rows x 4 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa_pairs = kb[[\"chunk_question\", \"chunk_title_text_concat\", \"chunk_id\", \"paragraph_id\"]]\n",
    "qa_pairs.columns = [\"question\", \"positive_chunk\", \"positive_chunk_id\", \"paragraph_id\"]\n",
    "qa_pairs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "GENERATIONS_SAVE_FILENAME =  f\"qa_pairs_{GENERATION_MODEL_NAME_OR_PATH}_num_questions_{len(qa_pairs)}_{DOMAIN}\"\n",
    "GENERATIONS_SAVE_FILENAME = re.sub(r'\\W+', '_', GENERATIONS_SAVE_FILENAME)\n",
    "GENERATIONS_SAVE_PATH = os.path.join(GENERATIONS_SAVE_DIR, f\"{GENERATIONS_SAVE_FILENAME}.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generated QA Pairs saved to /workspace/files/data/qa_pairs_meta_llama_Llama_2_13b_chat_hf_num_questions_300_BeIR_nfcorpus.csv\n"
     ]
    }
   ],
   "source": [
    "qa_pairs.to_csv(GENERATIONS_SAVE_PATH, index=None)\n",
    "print(f\"Generated QA Pairs saved to {GENERATIONS_SAVE_PATH}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
